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HUMAN  FACTORS  ENGINEERING 


LESSON  26:  MAINTAINABILITY,  OR  CAN  ANYONE  FIX  THIS? 


Hello  again,  and  welcome  back  to  your  course  on  Hunan  Factors 
Engineering.  If  you'll  think  back  a  few  lessons,  you'll  reneaber  that 
since  our  hero  I.  M.  Eager  'got  his  act  together*  and  cane  under  the 
tutelage  of  Professor  Ed  U.  Kator,  he  has  been  exposed  to  a  nunber  of 
important  huaan  factors  engineering  topics.  Along  with  Eager,  you  have 
studied  the  processes  of  systens  acquisition,  systens  analysis,  task 
analysis,  and  cost  analysis  and  have  learned  sone  of  the  lnportant  terns 
and  concepts  related  to  each  of  these  topics. 

In  this  lesson  you  will  learn  about  an  additional  area  of  concern  to 
human  factors  engineering,  nanely,  designing  for  naintainability.  In  order 
to  get  his  students  thinking  about  naintainability,  why  it's  a  hunan 
factors  problen,  and  what  variables  contribute  to  naintainability, 
Professor  Kator  asked  each  student  to  think  of  a  systen,  and  to  consider 
the  problens  that  night  arise  if  that  systen  were  designed  without 
consideration  for  its  naintainability.  Eager  naturally  thought  of  his 
super  chopper  and  focused  on  how  designing  for  naintainability  had  been  an 
lnportant  consideration  in  the  final  design  of  his  super  machine. 

Kator  then  used  this  question  and  the  resulting  discussion  as  a 
vehicle  to  begin  his  lecture  on  maintainability.  Kator  began  by  asking  his 
ctuderts  if  they  knew  for  sure  what  naintainability  is ,  and  what  types  of 
activities  are  Involved.  Ed  was  not  at  all  surprised  to  find  that  nany  of 
the  students  were  unsure  of  the  nature  and  scope  of  the  topic  they  had  been 
discussing  for  the  last  15  nlnutes. 

Once  again  Ed  U.  Kator  used  this  outcone  to  expertly  naneuver  the 
class  to  the  real  starting  point  of  this  lesson.  He  laid  out  the  exact 
plan  for  this  lecture  (and  incidentally,  we'll  also  follow  Ed's  structure 
in  presenting  the  information  to  you).  First,  we  will  look  at  sone  of  the 
basic  concepts  and  definitions  associated  with  this  topic;  nanely,  we  will 
discuss  the  similarities ,  differences,  and  relationships  between 
maintainability,  maintenance,  and  reliability.  If  you  are  at  all  fanlllar 
with  this  area,  we're  sure  you've  heard  sone  of  these  'buss  words.'  After 
we  define  each  of  these  concepts,  we'll  discuss  sone  of  the  typical 
maintainability  design  features.  Fron  there  you'll  deal  with  analysis  and 
allocation  of  maintenance  functions,  looking  at  things  such  as  the 
trade-off  between  costs,  flexibility,  reliability,  speed,  and  available 
manpower. 


(GO  ON  TO  THE  NEXT  PAGE) 


How  does  all  of  this  sound  so  far?  Pretty  tantalizing,  eh?  Well, 
hold  on  to  your  replaceable  parts  because  we're  not  through  yet.  Before  we 
complete  the  lesson,  we  want  you  to  become  familiar  with  the  concept  of 
maintainability  prediction.  In  addition,  we  want  to  show  you  how  to 
increase  operational  efficiency  by  focusing  on  methods  to  improve  the 
relationship  between  the  design  of  a  particular  maintenance  task  and  the 
skills  involved  in  performing  that  task.  Before  we  ramble  on  any  further, 
let's  get  to  the  task  at  hand — so  prepare  yourself  for  the  new  best-seller, 
'maintainability. ' 

The  increasing  complexity  of  modern-day  equipment— both  commercial  and 
military — has  resulted  in  new  types  of  engineering  problems.  These 
problems  have  increased  the  importance  of  designing  systems  so  as  to 
enhance  their  maintainability,  reliability,  and  high  performance. 
Historically,  design  engineers  have  approached  these  problems  by  focusing 
on  the  equipment  while  ignoring  the  'man'  aspects  of  the  man/machlne 
interface.  The  primary  concern  of  the  engineer  was  to  provide  hardware 
that  would  accomplish  the  specified  function  of  the  system.  This  is  not  to 
say  that  the  human  was  not  considered  at  all,  but,  rather,  that  little 
attention  was  focused  on  his  capabilities,  his  limitations,  or  other  'human 
factors'  in  the  design. 

In  the  preceding  paragraph  we  mentioned  terms  such  as  maintainability 
and  reliability.  Are  you  familiar  with  these  terms,  what  they  mean,  how 
they're  related,  and  why  they're  Important?  If  you're  not  now  familiar 
with  them,  before  this  lesson's  over  you'll  know  them  intimately.  If  you 
are  familiar  with  them,  we  hope  the  following  pages  will  introduce  you  to 
some  new  and  Important  facets  of  these  problems. 

Now  can  you  tell  us  your  definition  of  maintainability?  Which  of  the 
following  would  you  say  best  defines  maintainability? 

(1)  A  design  characteristic  which  is  the  probability  that  an  item  will  be 
retained  in  or  restored  to  a  specified  operating  condition  within  a  given 
period  of  time,  when  maintenance  is  performed  according  to  prescribed 
precedures  and  resources.  Turn  to  Page  18. 

(2)  All  actions  necessary  for  retaining  an  item  in  or  restoring  it  to  a 
specified  operating  condition.  Turn  to  Page  89. 

(3)  Both  of  these  answers  are  correct.  Turn  to  Page  71. 

(4)  Neither  of  these  answers  is  correct.  Turn  to  Page  43. 


From  Page  68 
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(1)  This  answer  is  Incorrect.  There  are  a  number  of  important  maintain¬ 
ability  areas  that  must  be  evaluated  in  terms  of  the  trade-offs  involved. 
Go  back  to  Page  68. 


From  Page  52 


(3)  Nope,  we  fooled  you.  This  answer  does  not  refer  to  what  we  would  do 
with  reliability  information.  It  gives  you  two  ways  of  determining  system 
complexity,  and  we  would  use  these  to  decide  on  a  course  of  action.  It  is 
the  course  of  action  itself  that  we  need  as  a  correct  answer.  Return  to 
Page  52. 


Frost  Page  47 


(4)  Look  back  at  your  assignment.  Only  one  of  these  answers  describes 
precisely  the  objective  of  phase  five.  Return  to  Page  47. 


I _ I 

(2)  Only  recently  have  computer  programs  been  used  as  a  technique  to 
Increase  skill  levels.  Return  to  Page  62. 


Proa  Page  13 


(2)  This  Is  only  one  source  by  which  you  could  recognize  the  existence  of  a 
hazard.  Go  back  to  Page  13. 


Proa  Page  24 


(4)  Look  a  bit  more  closely.  You’re  making  some  false  assumptions.  For 
Instance,  just  because  a  methodology  has  been  proven  valuable  in  industry, 
this  doesn't  necessarily  mean  it  will  be  available  to  the  military.  Return 
to  Page  24. 


Proa  Page  31 


(2)  Maybe  we  confused  you.  In  this  situation  the  sonarman's  responsibility 
is  to  detect  subs.  T.ere  was  a  signal  (sub),  but  the  response  given  was 
that  there  was  no  signal.  That  places  this  situation  in  the  cell  marked 
'miss.'  The  stimulus  was  'on'  but  the  response  was  'no.'  Return  to  Page 
31. 
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(2)  Very  good,  you  would  be  likely  to  give  a  large  number  of  responses, 
and,  because  there  was  no  penalty  for  false  alarms,  both  these  and  the 
number  of  hits  would  Increase. 

In  general,  If  the  payoff  Is  high  and  there  are  no  penalties  for 
incorrect  Judgments,  you  can  expect  an  observer  to  have  a  high  rate  of  hits 
and  false  alarms.  You,  as  an  experimenter,  can  change  the  proportion  of 
responses  which  occur  In  Figure  30.1  by  changing  the  payoff  and  penalty 
structure.  An  example  of  this  takes  us  back  to  our  dew  line  observer. 
Suppose  that  it  is  a  peacetime  situation  and  the  observer  really  doesn't 
expect  to  find  any  enemy  aircraft.  In  fact,  his  main  concern  is  that  he 
doesn't  Identify  a  TWA  flight  as  an  enemy  plane.  The  enemy  craft  Is  the 
signal  In  this  situation.  There  is  the  llkelhood  that  the  number  of 
correct  rejections  will  be  the  highest  of  all  the  cells  in  Figure  30.1.  If 
the  rate  of  presentation  remains  constant  (the  number  of  enemy  craft),  but 
the  expectancy  of  presentation  changes  (war  is  declared)  then  the  number  of 
false  alarms  will  increase. 

So  now  you  can  see  how  signal  detection  takes  Into  account  the 
observer' 8  biases  and  how  these  biases  Influence  the  observer's  responses. 
The  classical  psychophysical  methods  assumed  that  with  trained  observers 
these  biases  were  overcome  and  that  responses  were  influenced  only  by 
changes  In  the  stimulus. 

Okay,  so  far  we've  discussed  the  classical  methods  and  signal 
detection  theory.  These  methods  and  their  differences  are  nice  to  know, 
but  we  are  primarily  Interested  In  how  they  Impact  upon  the  human  factors 
specialist.  Why  would  you  use  a  classical  method  in  your  work? 

(1)  To  establish  the  observer's  bias  and,  therefore,  ensure  easy 
detectability.  Turn  to  Page  48. 

(2)  To  ensure  that  stimuli  are  below  the  lower  difference  level  and, 
therefore,  would  be  readily  detected.  Turn  to  Page  64. 

(3)  To  ensure  that  stimulus  values  are  above  the  lower  difference  level 
and,  therefore,  will  be  readily  detected.  Turn  to  Page  41. 


(1)  Absolutely  correct.  Hunan  error  does  account  for  the  largest 
percentage  of  accidents. 

As  was  mentioned  in  Lesson  3t  today  there  Is  a  concerted  effort  to  use 
hunan  factors  engineers  not  only  to  correct  faulty  design,  but  as  a  part  of 
the  original  design  process  as  well.  In  this  way,  tragic  errors  and 
potential  hasards  and  accidents  can  be  prevented.  Thus,  in  contributing  to 
the  design  of  a  system,  the  human  factors  engineer  should  help  to  provide 
conditions  which  optimize  operator  performance  and  protect  the  health  and 
safety  of  system  operators.  Of  course,  that  effort  must  be  done  within  the 
deslgn~to-cost  principles  discussed  In  Lesson  25  (affordability). 

As  you  have  seen  in  your  previous  lessons,  the  scope  of  human  factors 
engineering  encompasses  a  wide  range  of  system  safety  considerations. 
Areas  of  consideration  Include  such  variables  as  noise  levels  (Lesson  19), 
noxious  fumes  and  gases  (Lesson  18),  shock  and  vibration  (Lesson  15), 
warning  devices  and  labels  (Lesson  10  and  12),  as  well  as  extreme  climatic 
environments  (Lesson  17).  In  addition,  human  factors  engineers  evaluate 
system  safety  considerations  in  the  areas  of  fire  hazards,  emergency 
escapes,  and  electrical  and  mechanical  hazards.  The  military 
specifications  you  are  already  familiar  with,  such  as  MIL-HDBK-759 , 
typically  have  a  subsection  on  safety  and  tolerable  limitations  for  each 
factor  under  consideration.  MIL-STD-1472  has  an  additional  subsection 
devoted  to  hazards  and  safety  which  can  be  found  in  section  5.13  on  Page 
189. 

So,  if  the  basic  principles  of  human  factors  design  are  employed  when 
devising  a  new  system,  the  overall  safety  of  the  operator  will  be  taken 
into  account.  Why  then  do  you  suppose  that  this  lesson  has  been  Included 
in  your  human  factors  engineering  course? 

(1)  To  fill  in  space.  Turn  to  Page  20. 

(2)  To  reemphasize  the  findings  of  previous  lessons.  Turn  to  Page  49. 

(3)  To  Introduce  you  to  a  method  for  evaluating  safety  conditions  in  an 
existing  system.  Turn  to  Page  12. 


& 


From  Page  81 


(3)  While  it's  true  that  interchangeability  is  an  Important  maintainability 
feature,  it  should  not  be  concluded  that  it  is  always  the  most  important 
design  feature.  Return  to  Page  81. 


o 


From  Page  15 


(3)  You're  incorrect.  Initial  concerns  in  the  maintainability  area  rested 
on  design  improvements  only.  Return  to  Page  15. 
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From  Page  98 


(4)  You're  correct  in  saying  the  organizational  differences  must  be 
considered,  but  this  fact  prohibits  generalization  to  a  great  extent. 
Return  to  Page  98. 
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(3)  This  isn't  the  only  source  you  can  use  to  Identify  a  hazard.  Return  to 
Page  13. 


From  Page  9 


(1)  In  general,  this  Is  true.  Given  a  good  program  and  the  right  kind  of 
training  requirements,  OJT  is  the  most  efficient  type  of  training.  But 
there  are  Instances  when  OJT  won't  work  very  well  (e.  g.,  a  jet  pilot). 
Return  to  Page  9  and  try  again. 


From  Page  30 


(4)  Auditory  threshold  is  only  somewhat  correct,  since  our  example  was 
about  hearing.  However,  there  is  an  answer  that  is  correct  without  regard 
to  the  sensory  modality  In  question.  Return  to  Page  30. 


Proa  Page  95 


(1)  Very  good.  Since  the  military  must  always  be  prepared  for  wartime,  in 
developing  manpower  data,  you  must  assume  this  type  of  scenario. 

The  third  source  document  of  Importance  is  the  reliability  analysis. 
This  provides  computations  and  statistical  analyses  compiled  by  reliability 
engineers.  This  document  provides  information  on  elements  which  are 
expected  to  fall  prior  to  the  scheduled  time  of  replacement.  Analysis  of 
this  data  provides  the  human  factors  specialist  with  information  regarding 
maintenance  load  and  its  effects  on  operators. 

A  final  source  of  information  is  operating  manuals  for  similar 
systems.  By  examining  these  documents  and  by  talking  with  persons 
experienced  in  similar  systems,  the  human  factors  specialist  can  get  a  good 
idea  of  the  requirements  of  the  system  with  which  he  is  working. 

If,  after  examining  all  these  documents,  you  discover  that  individuals 
already  exist  to  operate  and  maintain  the  system,  then  your  job  as  a  human 
factors  engineer  is  much  easier.  But  more  often  than  not,  with  a  new 
system,  skilled  people  will  not  be  available  and  training  will  be  required 
in  one  form  or  another.  While  you  as  a  human  factors  specialist  will  not 
be  directly  involved  in  training,  an  understanding  of  the  training  is 
important . 

There  are  several  techniques  which  can  be  used  for  training.  We  will 
examine  four  general  methods.  These  are: 

(1)  On-The-Job  Training  (OJT) 

(2)  Classroom  Training 

(3)  Computer-Aided  Instruction  (CA1) 

(4)  Team  Training 

On-the-job  training  (OJT)  is  one  of  the  most  widely  used  training 
methods  found  in  the  military.  Its  name  is  fairly  descriptive,  but  its 
real  advantages  may  be  more  difficult  to  understand.  You've  heard  about 
OJT.  What  do  you  think  is  its  greatest  advantage? 

(1)  Since  it  is  done  'on-the-job'  it  is  always  more  efficient.  Turn  to 
Page  8. 

(2)  The  trainee  has  little  problem  transferring  what  he  learns  from  the 
training  situation  to  the  real  job.  Turn  to  Page  24. 

(3)  It  requires  less  effort  putting  together  OJT  programs,  since  those 
conducting  the  training  do  the  work  every  day.  Turn  to  Page  47. 

(4)  All  of  the  answers  presented  here.  Turn  to  Page  75. 


From  Page  37 


(2)  Good  show,  you've  done  It  again.  General  hazard  analysis  does  consist 
of  these  three  phases. 

Generally,  systems,  by  their  very  nature,  possess  hazards.  For 
example,  temperature  extremes  occur  in  the  engine  and  boiler  rooms  of  a 
naval  vessel.  Power  sources  necessary  for  computer  facilities,  as  well  as 
nuclear  reactors  on  some  types  of  vessels,  are  also  sources  of  hazards. 
Add  to  these  things  the  fact  that  the  ship  is  moving  back  and  forth  and/or 
up  and  down  in  the  middle  of  the  sea,  and  you  can  see  that  there  are  plenty 
of  reasons  why  hazard  analysis  is  a  necessity.  People  are  always  a  part  of 
any  system  and  they  are  not  infallible.  Because  of  this,  you  can  never 
expect  to  eliminate  all  hazards  and  prevent  all  accidents.  Therefore,  as  a 
human  factors  specialist,  you  will  want  to  eliminate  as  many  hazards  as 
possible  and  concentrate  on  those  conditions  which  are  most  severe.  It  is 
necessary,  therefore,  not  only  to  identify  the  existence  of  hazards  but  to 
evaluate  the  hazards  found. 

This  evaluation  will  Involve  estimating  the  probable  seriousness  of  a 
potential  accident  as  well  as  estimating  the  probability  of  occurrence  of 
the  accident.  For  example,  if  a  heat  source  is  located  close  to  flammable 
material,  there  is  a  high  probability  that  an  accident  will  occur.  On  the 
other  hand,  if  the  flammable  material  is  placed  in  a  metal  container  away 
from  the  heat  source,  there  is  a  low  probability  that  an  accident  will 
occur. 


In  addition  to  estimating  the  probability  of  an  accident,  the 
evaluation  of  hazards  should  also  include  an  estimate  of  the  severity  of 
any  damage  or  loss  which  would  result  if  the  hazard  led  to  an  accident.  In 
the  heat  sgurces  example,  tremendous  losses  might  occur  if  the  flammable 
material  caught  fire  in  a  heavily  populated  area.  This  evaluation  will 
allow  you  to  compare  one  hazard  with  another  in  order  to  determine  which 
potentially  hazardous  condition  should  be  corrected  first.  It  will  also 
allow  you  to  establish  a  priority  scheme  to  use  in  deciding  whether  or  not 
it  would  be  cost-effective  to  try  to  correct  existing  conditions. 

(GO  ON  TO  THE  NEXT  PAGE) 
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From  Page  10 
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Now  you  are  ready  to  look  at  the  evaluation  phase  In  detail.  Remember 
that  the  purpose  of  hazard  analysis  is  to  suggest  or  establish  a  plan  of 
action  which  will  lead  to  hazard  elimination  or  reduction.  Without  a 
method  of  distinguishing  among  various  hazards,  your  plan  of  action  may  not 
be  appropriate.  Figure  27.3  in  your  supplement  is  an  example  of  a  data 
form  which  shows  four  types  of  evaluation  that  can  be  conducted  for  each 
hazard.  The  four  evaluation  factors  are  severity,  probability,  cost,  and 
action.  The  categories  under  each  factor  are  arbitrary,  but  the  ones  you 
choose  should  cover  the  range  shown  in  Figure  27.3.  By  using  a  recording 
method  such  as  this  one,  you  can  help  reduce  the  individual  pieces  of 
information  gathered  into  groups  of  similar  types  of  information.  Of  the 
groupings  shown,  which  is  the  most  important? 


(1)  Cost.  Turn  to  Page  93. 

(2)  Probability.  Turn  to  Page  99. 

(3)  Severity.  Turn  to  Page  92. 

(4)  Action.  Turn  to  Page  40. 


From  Page  22 


(3)  This  is  a  tough  one.  A  job  analysis  is  required,  but,  although  funding 
is  a  consideration,  it  is  not  a  formal  part  of  this  phase.  Return  to  Page 
22. 


-- 
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(3)  Right,  we  will  review  a  method  of  safety  analysis,  more  accurately 
hazard  analysis,  of  the  entire  system. 

It  is  hard  to  give  an  adequate  definition  of  an  accident,  but  we,  as 
human  factors  specialists,  are  concerned  with  the  problem  of  accidents. 
Generally  speaking,  an  accident  is  an  event  or  occurrence  which  is  both 
undesirable  and  unexpected.  It  results  from  faulty  equipment  or  human 
performance  errors.  When  an  accident  occurs,  there  may  or  may  not  be 
injury  to  either  the  person  or  the  equipment,  but  there  usually  is  a 
disruption  of  the  normal  work  routine.  Accidents  almost  always  result  in 
time  delays  and  reduced  production. 

In  this  lesson  we  will  look  at  accidents  as  events  that  occur  when 
either  the  people,  machines,  or  materials  are  out  of  control.  In  so  doing, 
we  can  look  at  the  system  in  terms  of  control.  By  analyzing  the  system,  we 
should  be  able  to  determine  which  part  is  deficient  so  that  improvements 
can  be  made.  Let  us  emphasize  the  idea  that  human  performance  errors  may 
largely  (not  entirely)  be  'designed  out'  of  hardware;  also,  lots  of  them 
may  be  'designed  in.'  By  using  the  systems  approach,  it  is  possible  to 
plan  ahead  so  that  a  good  deal  of  the  potentially  hazardous  situations  will 
be  eliminated  in  the  design  of  the  system.  Therefore,  costs  for 
breakdowns,  change,  and  reinstallation  will  be  saved. 

One  way  of  analyzing  the  system  is  by  conducting  a  hazard  analysis. 
Hazard  analysis  consists  of  three  parts:  (1)  identifying  the  hazard,  (2) 
evaluating  the  hazard,  (3)  selecting  the  best  countermeasures  for  the 
particular  situation. 

The  first  step  in  eliminating  a  hazard  is  recognizing  that  there  is 
one;  that  is,  to  identify  the  existence  of  the  hazard.  This  can  be  done  in 
several  ways.  Many  sources  of  potential  danger  can  be  pinpointed  just  by 
looking  at  the  system  itself.  For  example,  the  more  typical  design  aspects 
which  are  involved  in  accidents  can  often  be  found  in  the  accident 
prevention  manual  for  industrial  operations.  This  manual  is  published  by 
the  National  Safety  Council  and  contains  a  machine  safety  checklist,  which 
you  should  become  familiar  with.  As  an  example  of  one  item  on  the 
checklist,  notice  the  corners  of  the  table  or  desk  at  which  you  usually 
work.  Are  the  corners  sharp  or  rounded?  If  they  are  sharp,  then  they  are 
a  potential  source  of  accidents,  to  say  nothing  of  pain  when  you  come  into 
rapid  contact  with  them,  huh?  Which  of  the  following  do  you  think  is  also 
a  general  source  used  to  recognize  a  hazard? 


(GO  ON  TO  THE  NEXT  PAGE) 
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From  Page  12 


(1)  Tripping  over  a  raised  piece  of  rug.  Turn  to  Pagt  82. 

(2)  Historical  evidence  that  shows  that  accidents  occur  at  a  particular 
location.  Turn  to  Page  4. 

(3)  Experienced  craftsmen  telling  you  about  hazards  in  their  work  area. 
Turn  to  Page  8. 

(4)  All  of  these  are  ways  to  recognize  and  Identify  hazards.  Turn  to  Page 
16. 


From  Page  88 


4s* 

r*  (2)  You're  correct  when  you  say  that  this  answer  is  important  for  designing 

for  ease  of  replacement,  but  there  is  a  more  correct  response.  Return  to 
Page  88. 


From  Page  72 


w  (3)  Determining  whether  or  not  a  given  job  can  be  performed  is  the  ultimate 

n  goal  of  this  series  of  analyses,  but  it  is  not  the  main  goal  of  a  job 

analysis.  Return  to  Page  72  and  try  again. 


(1)  Good  show.  Through  the  1950s,  Intensive  training  was  promoted  as  a 
method  of  increasing  maintainability  effectiveness. 


With  the  increased  complexity  of  equipment  and  the  realization  that 
training-to-improve-maintalnability  had  certain  limitations,  other  methods 
of  improving  maintainability  were  examined. 

Technical  manuals  were  devised  to  provide  quite  explicit  equipment 
operation  procedures,  including  where  possible  trouble  spots  could  occur. 
These  technical  manuals  have  proven  helpful,  but  equipment  can  fail  in  many 
ways,  and  a  good  deal  of  'logical  troubleshooting'  is  also  called  for  when 
using  the  technical  manual  maintainance  procedures.  Logical 
troubleshooting  combines  a  'high  degree  of  problem-solving  ability'  and  'a 
game  of  probabilities.' 

Computer  programs  for  maintenance  were  also  proposed  and  used  to  aid 
maintainability.  The  problems  found  here  were  much  the  same  as  with  the 
technical  manuals;  namely,  the  practical  impossibility  of  dealing  with  all 
possible  combinations  of  faulty  connections.  Currently,  all  these 
techniques  are  being  used  to  increase  maintainability.  In  addition, 
maintainability  prediction  is  considered  to  be  an  Important  method  of 
Increasing  maintainability,  and  can  predict  the  likelihood  of  a  certain 
fault  occurring  at  a  particular  point  and  a  particular  time. 

An  example  of  using  computer  programs  to  judge  the  readability  of 
technical  manuals  can  be  found  in  HEL  TM  4-80.  Ibis  manual  presents  a 
computer  program  which  will  analyze  the  text  in  a  number  of  ways,  such  as: 

(1)  Number  of  words  with  three  or  more  syllables. 

(2)  Number  and  average  length  of  sentences. 

(3)  Average  syllables  per  word* 

Finally,  this  program  provides  a  reading  grade  level  score.  Now,  if 
you  know  that  the  average  maintenance  technician  has  a  reading  level  of  X 
and  the  tech  manual  which  must  be  used  is  written  at  level  X  +  5,  then  you 
know  you  need  to  redo  the  tech  manual. 

(GO  ON  TO  THE  NEXT  PAGE) 


From  Page  25 


(1)  This  answer  does  require  an  estimation  procedure,  but  not  an  Interval 
estimation.  Return  to  Page  25. 
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We  hope  this  discussion  of  maintainability  has  shown  the  strong 
relationship  that  exists  between  system  design  characteristics  and 
technician  skill  level.  Which  of  the  following  phrases  would  be  most 
descriptive  of  this  relationship? 

(1)  The  designer  must  take  into  consideration  the  skill  level  of  the  person 
who  will  be  charged  with  the  task  of  maintaining  the  equipment.  It  would 
be  ludicrous  to  make  all  maintenance  steps  incomprehensible  to  the 
malntainers.  Turn  to  Page  52. 

(2)  The  designers  need  simply  to  worry  about  the  electronic  or  mechanical 
aspects  of  their  machinery;  technicians  at  all  maintenance  levels  will 
possess  the  same  skills.  Turn  to  Page  32. 

(3)  Maintainability  has,  in  the  past,  always  been  largely  concerned  with 
the  technician's  skill  level.  Turn  to  Page  7. 

(4)  All  of  these  statements  are  accurate.  Turn  to  Page  45. 


From  Page  51 


(2)  You're  only  partially  right.  Individualized  training  and  immediate 
feedback  are  two  aspects  of  CAI,  but,  unfortunately,  cost  is  sometimes  a 
prohibitive  factor.  Return  to  Page  51. 


From  Page  69 


(4)  This  is  a  tough  one.  But  if  you  examine  the  answer  closely,  you'll  see 
that  only  one  term  really  answers  the  question.  Return  to  Page  69. 


From  Page  13 


(4)  Very  good.  In  addition  to  looking  at  the  system  itself,  there  are 
other  sources  by  which  you  could  identify  a  hazard.  These  are  historical 
records,  safety  checklists,  and  other  information  obtained  from  general 
investigations . 

Historical  data  may  indicate  that  accidents  are  common  at  a  given 
location,  but  a  trained  systems  analyst  can  also  use  his  or  her  judgment  to 
identify  additional  sources  of  hazard  locations. 

Nov,  we  have  mentioned  historical  records,  but  what  do  they  consist 
of?  Private-sector  industries  are  required  by  regulation  of  OSHA 
(Occupational  Safety  and  Health  Act)  to  maintain  accident  reports.  OSHA 
regulations  require  that  the  type  of  accident,  its  location,  and  its  result 
(in  the  form  of  measures  of  such  things  as  the  number  of  days  lost  from 
work)  be  recorded.  Of  course,  in  the  military  setting,  OSHA  regulations 
are  not  binding.  However,  do  you  think  our  military  services  are  more  or 
less  stringent  in  their  safety  programs  than  that  required  of  private 
sector  industries? 

(1)  More.  Turn  to  Page  79. 

(2)  Less.  Turn  to  Page  70. 

(3)  In  between.  Turn  to  Page  87. 


HUMAN  FACTORS  ENGINEERING 


LESSON  27:  HAZARD  ANALYSIS,  OR  A  STITCH  IN  TIME 


Hello,  this  is  Lesson  27  of  your  Human  Factors  Engineering  Course.  In 
this  lesson  you-Hrlll  learn  about  analyzing  for  safety  by  Investigating  the 
hazards  which  may  be  present  in  a  system.  You  also  will  be  referred  to  a 
summary  of  safety  design  principles  in  MIL -STD  1742 . 

Before  we  get  into  hazard  analysis,  let's  recap  a  bit  so  that  this 
aspect  of  Human  Factors  Engineering  is  put  into  perspective.  Remember  that 
the  central  focus  of  human  factors  is  the  concern  for  the  operator  in  the' 
design  of  machines  and  environments.  With  this  consideration  in  mind,  our 
objectives  are  to  enhance  the  effectiveness  of  an  individual's  performance. 
Our  primary  objective  is,  of  course,  to  enhance  human  values.  Values  such 
as  health  and  safety  are  important  in  and  of  themselves,  but,  they  are  also 
important  from  a  pragmatic  point  of  view.  If  the  people  working  and  living 
in  a  system  lack  these  human  values,  then  their  effective  functioning  will 
be  diminished.  Therefore,  a  fair  part  of  the  human  factors  engineer's  job 
is  to  ensure  that  the  system  is  safe  for  its  operators. 

Everything  you  have  learned  so  far  in  this  course  has  been  aimed  at 
these  objectives.  Even  our  oft-repeated  statement  about  relating  the 
human's  capabilities  and  limitations  to  the  requirements  of  the  task  has 
these  objectives  in  mind.  After  all,  if  the  task  requirements  overreach 
the  person's  capabilities,  the  desired  outcome  will  not  be  accomplished  and 
an  unsafe  situation  may  develop.  If  you  recall,  this  was  what  Lesson  3, 
'Tragic  Mistakes  and  Positive  Consequences,'  was  all  about. 

In  that  lesson  we  discussed  the  major  cause  of  most  accidents.  Do  you 
recall  what  factor  was  identified  as  the  cause  of  most  accidents? 

(1)  Human  error.  Turn  to  Page  6. 

(2)  Equipment  malfunction.  Turn  to  Page  92. 

(3)  Gremlins  in  the  system.  Turn  to  Page  78. 


From  Page  2 


(1)  Very  good,  you're  absolutely  right.  Maintainability  is  concerned  with 
implementing  principles  basic  to  future  equipment  repair. 

Hell,  that  wasn't  exactly  the  easiest  question  to  answer,  was  it?  He 
hope  you  didn't  think  we  were  trying  to  trick  you  with  the  other 
definition,  which  seems  equally  plausible.  Rather,  we  were  trying  to  make 
a  stong  point  for  demonstrating  the  difference  between  maintainability  and 
maintenance.  There  is  a  distinction  between  the  two  words  as  each  term  is 
used  in  engineering  design.  While  'maintainability  engineering'  is 
concerned  with  equipment  design  fabrication,  or  developement  so  as  to 
facilitate  future  equipment  repair,  'maintenance  engineering'  is  concerned 
with  the  technical  problems  of  keeping  a  system  in  an  operational  state  or 
repairing  a  malfunctioning  unit  once  the  equipment  is  in  use.  The 
maintainability  engineer  directly  influences  the  effectiveness  cf  the 
maintenance  engineer.  Conversely,  the  maintenance  engineer  will  affect  the 
future  maintainability  principles  derived  and  implemented  by  the  designer. 

You  should  be  aware  that  these  terms,  as  well  as  others  related  to  the 
whole  area  of  maintainability,  are  defined  in  MIL-STD-721.  In  addition, 
the  whole  topic  of  maintainability  is  discussed  in  MIL-STD-1472 ,  so  keep 
these  in  mind  as  good  reference  sources.  Also,  your  supplement  has  two 
sources  which  apply  to  maintenance.  One  is  a  DID  and  the  othdr  is  LSAR 
which  you  saw  in  Lesson  24.  After  the  lesson  is  over,  we  suggest  you  look 
at  these  two  documents. 

An  Important  concept  to  be  considered  in  designing  for  maintainability 
is  'equipment  reliability.'  Based  on  what  you've  learned  about 
maintainability  and  maintenance  already,  what  would  you  think  reliability 
refers  to? 

(1)  The  numerical  probability  of  failure-free  system  (or  equipment) 
operation  during  a  prescribed  time  period  under  certain  specified 
conditions  of  performance  and  environment.  Turn  to  Page  76. 

(2)  The  probability  that  an  item  will  be  retained  or  restored  to  a 
specified  condition  within  a  given  period  of  time,  when  maintenance  is 
performed  under  prescribed  conditions.  Turn  to  Page  64. 

(3)  All  actions  necessary  for  retaining  an  item  in  or  restoring  it  to  a 
specified  condition  after  a  breakdown  has  occurred.  Turn  to  Page  80. 


(4)  You're  exactly  right.  All  the  answers  presented  here  are  incorrect. 
They  make  generalizations  which  are  too  broad.  Simulation  is  the  preferred 
choice  when  we  talk  about  some  sophisticated  equipment,  but  not  for  more 
simple  tasks.  There  are  times  when  cost  will  prohibit  the  use  by 
simulators.  As  far  as  realistic  replication,  don't  forget  OJT. 

To  conclude  these  two  lessons  on  training,  we  will  review  some  key 
concepts  of  which  you  must  be  aware.  Zn  the  course  of  your  job,  you  will 
hear  certain  terms,  and  this  brief  overview  will  allow  you  to  understand 
better  the  topics  being  discussed. 

Perhaps  the  most  critical  issue  is  that  of  fidelity.  There  are  two 
types  of  fidelity  that  are  important  in  the  realm  of  simulators:  physical 
and  psychological  (or  functional).  Physical  fidelity  is  the  degree  of 
which  a  simulator  represents  the  real  world  of  the  operational  equipment. 
Psychological  or  functional  fidelity  is  the  degree  to  which  a  simulator 
reproduces  in  the  training  tasks  the  behavioral  processes  that  are 
necessary  to  perform  a  job.  With  this  Introduction  to  fidelity,  try  your 
hand  at  the  following.  As  a  training  analyst  concerned  with  teaching 
decision-making  abilities,  which  is  more  important? 

(1)  Psychological  or  functional  fidelity.  Turn  to  Page  63. 

(2)  Physical  fidelity.  Turn  to  Page  78. 

(3)  Psychological  fidelity  is  more  important,  but  you  cannot  have 
psychological  fidelity  unless  you  have  a  high  degree  of  physical  fidelity. 
Turn  to  Page  50. 

(4)  None  of  these  is  correct.  You  cannot  use  simulators  when  dealing  with 
decision  making  abilities.  Turn  to  Page  56. 


From  Page  52 


(4)  Boo!  Sure  is  a  lot  of  pie  in  the  sky  around  here.  In  the  best  of  all 
possible  worlds  we  might  have  a  system  comprised  of  replaceable  units.  But 
even  then,  there  are  bound  to  be  human  errors  and  you  just  made  one.  Try 
again.  Return  to  Page  52. 


From  Page  6 


(1)  Really!  By  selecting  this  answer  you've  cut  us  to  the  quick.  We  will 
forgive  you,  however,  because  we  are  really  nice  people  and  we  know  you 
couldn't  mean  it.  Could  you?  Return  to  Page  6  and  try  again. 


(1)  You're  partially  right.  A  job  analysis  must  take  place  during  this 
phase,  but  course  outlines  are  not  developed  until  later  in  the  cycle. 
Return  to  Page  22. 


(2)  Very  good*  This  was  an  example  of  an  absolute  threshold. 

An  absolute  threshold  Is  the  upper  ci  luster  value  of  the  stimulus 
range  which  can  be  detected  50  percent  of  the  time.  You  may  see  it 
referred  to  in  research  studies  as  an  absolute  Ilmen  (AL)  or  reiz  Ilmen 
(RL;  from  the  German). 

Another  type  of  threshold  value  which  is  Important  for  human  factors 
specialists  is  the  difference  threshold  (DL).  The  DL  can  also  be  referred 
to  as  a  just  noticeable  difference  (JND).  It  is  the  minimum  amount  of 
stimulus  that  needs  to  be  added  or  subtracted  in  order  for  an  individual  to 
detect  the  face  that  the  stimulus  has  changed.  While  there  is  a  technical 
difference  between  the  two  terms,  it  won't  be  necessary  for  us  to  be 
concerned  with  it  in  this  lesson.  For  example,  suppose  a  series  of  tones 
is  to  be  presented:  you  need  to  determine  how  much  the  frequency  of  a 
given  tone  must  be  increased  or  decreased  before  the  individual  will  be 
able  to  notice  a  change  in  pitch.  This  amount  of  change  (measured  in  Hz  in 
this  case)  is  called  a  DL. 

There  are  three  classical  measurement  procedures  that  may  be  used  to 
determine  an  individual's  threshold.  These  are  the  method  of  limits,  the 
method  of  adjustment,  and  the  method  of  constant  stimuli. 

The  method  of  limits  or  just  noticeable  differences  provides  the  most 
direct  method  of  determining  a  threshold.  This  method  can  be  used  to 
determine  both  RL  and  DL;  however,  it  is  primarily  used  in  determining  an 
RL.  In  order  to  determine  an  RL,  the  experimenter  should  present  several 
series  of  trials  to  each  observer.  In  these  trials  different  stimulus 
values  will  be  presented  in  alternating  ascending  and  decending  order.  On 
each  series  of  trials  there  is  a  point  at  which  the  observer's  responses 
change.  For  example,  in  an  ascending  series  of  500  Hz  auditory  stimuli, 
the  observer  may  indicate,  'no  I  can't  hear  the  tone'  when  it  is  presented 
at  10,  11,  12,  and  13  Hz;  but,  'yes  I  can  hear  it'  at  14  Hz  and  up.  By 
presenting  ascending  and  descending  series  of  trials,  the  experimenter  is 
able  to  take  into  account  the  variability  of  the  subject's  responses. 
Finally,  the  RL  is  determined  to  be  the  average  stimulus  values  at  which 
this  change  in  response  occurs. 

(TURN  TO  PAGE  67) 
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HUMAN  FACTORS  ENGINEERING 


LESSON  29:  DOES  THE  TRAINING  WORK? 


Welcome  back  to  your  Human  Factors  Engineering  Course.  As  you  recall, 
we're  In  the  middle  of  reviewing  a  course  which  Lt.  I.  M.  Eager  attended. 
The  professor,  Ed  U.  Kator,  was  involved  in  a  discussion  of  personnel 
selection  and  training.  If  you  recall,  at  the  end  of  the  last  class  both 
you  and  Eager  were  given  an  assignment.  The  reading  dealt  with  the 
instructional  systems  development  (ISD)  technique  for  training  course 
development.  As  a  review  of  this  reading  and  an  introduction  to  the 
material  to  be  presented  in  this  lesson,  answer  the  following  questions. 

The  ISD  system  consists  of  five  separate  phases.  The  first  phase  is 
called  'analyze.'  What  are  the  main  functions  of  this  phase? 

(1)  To  analyze  the  job  in  question  and  develop  a  course  outline  for 
required  training.  Turn  to  Page  20. 

(2)  To  analyze  the  job  under  consideration  and  decide  what  training  is 
required.  Turn  to  Page  47. 

(3)  To  analyze  job  requirements  and  ascertain  funding  requirements.  Turn 
to  Page  11. 


From  Page  81 


(1)  Ease  to  accessibility  cannot  be  noted  as  the  most  important  design 
feature  without  other  information  on  the  type  of  design  the  HFE  specialist 
is  dealing  with.  Return  to  Page  81. 


(3)  You’d  better  review  the  definitions  again.  Internal  validity  concerns 
Itself  with  the  training  environment  only.  Return  to  Page  55. 


(2)  When  the  observer  is  adjusting  a  light  source  to  where  he/she  'feels' 
It  is  equal  to  another  source,  he/she  cannot  be  said  to  be  making  an 
objective  judgment.  Return  to  Page  67. 


From  Page  9 
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(2)  Very  good*  The  problem  of  transferring  knowledge  from  one  environment 
to  another  Is  definitely  minimized  when  OJT  is  conducted  properly. 

With  OJT  you  have  to  be  careful.  Too  often  it  is  used  as  the  only 
training  program.  As  with  any  successful  program,  effort  must  be  put  forth 
to  obtain  successful  results.  Merely  having  one  worker  show  the  other  the 
ropes  is  not  a  comprehensive  OJT  program.  Most  often  OJT  is  supplemented, 
or  it  is  used  to  supplement  other  training  methods.  This  is  not  to  say 
that  OJT  in  itself  cannot  be  used,  but  with  most  major  military  systems, 
classroom  training  is  also  required. 

Training  with  simulators  may  be  considered  an  extension  of  the  OJT 
method,  even  though  it  is  net,  strictly  speaking,  'on-the-job.*  The 
simulator  provides  a  realistic  training  environment  which  is  often  less 
expensive  than  the  real  thing  and  safer  in  case  the  student  makes  an  error. 
Aircraft  simulators  are  excellent  examples.  More  often  than  not,  military 
training  combines  simulator  training  with  formal  classroom  presentations. 

There  is  another  method  the  military  uses  to  teach  equipment  operation 
and  maintenance  which  uses  the  OJT  philosophy.  In  the  Army  it's  called 
skilled  performance  aids  (SPAS);  in  the  Navy,  job  performance  aids  (JPAS). 
SPAS  and  JPAS  are  documents,  devices,  guides,  or  tools  which  supply 
information  and  guidance  to  aid  the  job  Incumbent  in  performing  a  task 
while  on  the  job.  While  JPAS  and  SPAS  are  similar,  SPAS  are  a  bit  more 
comprehensive  at  this  time,  so  we'll  emphasize  them.  As  time  goes  by,  both 
programs  will  more  nearly  reflect  each  other. 

The  skill  performance  aids  package  is  a  change  from  the  traditional 
teaching  of  theory  to  a  'hands  on'  form  of  training.  As  a  program,  SPAS 
provide  for  the  simultaneous  development  of  the  hardware  item,  the 
organization,  and  the  training,  with  each  element  supporting  and 
supplementing  the  other. 

Why  do  you  think  the  Army  is  so  excited  about  SPAS? 

(1)  Since  today's  soldier  is  highly  motivated  to  classroom  training,  the 
Army  feels  the  SPAS  will  meet  with  the  same  enthusiasm.  Turn  to  Page  80. 

(2)  Since  the  SPAS  methodology  has  proven  Itself  in  industry,  the  Army 
feels  certain  it  will  work  for  the  military.  Turn  to  Page  63. 

(3)  Because  this  method  concerns  itself  with  the  hardware  system,  the 
soldier,  and  the  cost.  Turn  to  Page  31. 

(4)  All  of  the  answers  listed  here.  Turn  to  Page  4. 
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(1)  Very  good.  The  production  procedures  require  that  the  observer 
actually  manipulate  the  stimulus  (as  In  the  classical  method  of  adjustment) 
until  he  produces  a  value  requested  by  the  experimenter. 

So  you  can  see  that  one  way  of  classifying  these  scaling  methods  Is  by 
what  they  require  of  the  subject.  Estimation  procedures  require  verbal 
responses  that  estimate  stimulus  values,  and  the  production  procedures 
require  an  action  on  the  part  of  the  subject  to  produce  specified  stimulus 
values . 

A  second  way  to  classify  the  scaling  methods  is  by  the  types  of 
judgments  required  of  the  subject.  He  may  be  required  to  judge  the 
difference  In  sensations  or  the  absolute  magnitude  of  sensation.  In  some 
cases  relative  judgment  of  magnitude  will  be  required.  We  will  now  discuss 
four  types  of  estimation  scales.  But  remember  that  similar  types  of  scales 
are  based  on  production  methods  as  well. 

(1)  Category  estimation  (or  production)  -  In  this  method  the  subject 
is  asked  to  Indicate  which  stimuli  belong  In  which  categories. 

(2)  Interval  estimation  (or  production)  -  this  method  requires  the 
subject  to  judge  the  Intervals  of  sensation  which  separate  stimuli. 

Before  we  continue  with  the  last  two  methods,  let's  create  a  situation 
and  ask  a  question.  You,  as  a  human  factors  specialist,  are  designing  a 
backpack  to  be  worn  during  field  manuevers.  You  have  four  different  packs 
which  have  already  been  developed,  and  each  of  these  packs  has  a  different 
weight  and  a  different  design.  You  want  the  users  to  have  the  pack  which 

feels  the  best  and  you  suspect  that  this  might  not  necessarily  be  the  pack 

which  weighs  the  least.  You  ask  a  group  of  users  to  try  on  each  pack. 
After  they  have  done  this,  you  ask  them  to  indicate  which  design  is  poor, 
good,  okay,  or  great.  Which  of  the  methods  presented  in  this  lesson  have 
you  used? 

(1)  Interval  estimation.  Turn  to  Page  14. 

(2)  Category  estimation.  Turn  to  Page  42. 

(3)  Method  of  limits.  Turn  to  Page  75. 

(4)  Signal  detection.  Turn  to  Page  90. 


From  Page  71 


(4)  Exactly  right*  Although  the  second  stimulus  was  actually  one  half  the 
first,  before  we  can  determine  assigned  values,  we  need  to  know  how  this 
signal  was  perceived  by  the  individuals* 

In  the  estimation  procedures,  a  large  number  of  trials  would  be  given 
so  that  all  the  stimulus  values  would  be  presented  several  times.  The 
average  response  of  each  subject  would  be  taken  for  each  stimulus  value. 
In  this  way,  a  scale  could  be  derived  for  each  individual.  Eventually,  the 
average  of  all  of  a  group  of  individuals'  responses  for  each  value  would  be 
taken.  He  recommend  that  you  take  a  look  at  any  Introductory  text  which 
deals  with  psychophysical  scaling  methods.  They  can  be  an  invaluable  aid 
when  making  decisions  as  to  design  features  which  require  sensory  judgments 
on  the  part  of  the  users.  They  can  help  you  make  decisions  as  to  how 
bright,  loud,  soft,  etc.  a  stimulus  must  be  to  be  differentiated  from  other 
values . 

Hell,  you've  learned  a  great  deal  in  this  lesson,  as  did  I.  M.  Eager 
when  he  took  the  course.  In  fact,  after  this  particular  lesson.  Eager  used 
the  magnitude  estimation  method  to  find  out  which  wattage  his  emergency 
light  had  to  be  in  order  for  it  to  be  perceived  as  twice  as  bright  as  other 
lights  on  the  Instrument  panel.  He  also  used  the  scaling  methods  to 
evaluate  how  well  people  like  the  food  in  the  cafeteria,  but  we'd  rather 
not  go  into  that,  if  you  don't  mind. 

So  long  for  now,  but  see  you  In  our  next  two  exciting  lessons  on 
designing  experiments.  Ta,  Ta. 


From  Page  51 


(1)  Exactly  right.  In  addition  to  the  individualized,  immediate  feedback 
aspects  described  previously,  the  fact  that  the  instructor  is  freed  to 
concentrate  on  specific  problems  makes  CAI  an  often-preferred  method. 
Perhaps  the  best  advantage  of  all  is  being  able  to  avoid  having  to  get  all 
the  students  together  at  one  time  and  place.  This  course  is,  of  course,  an 
outstanding  example  of  CAI.  Right?  Right! 

The  final  method  of  training  we  will  examine  is  team  training.  Quite 
simply,  team  training  can  be  defined  as  training  a  group  of  personnel  to 
perform  related  or  integrated  functions  as  a  team.  Since  most  highly 
complex  systems  require  a  number  of  people  to  work  together,  the  need  for 
team  training  is  obvious.  However,  research  has  shown  that  to  be 
successful,  team  training  must  be  integrated  with  the  individual 
instruction. 

Well,  you've  completed  a  very  brief  review  of  personnel  selection  and 
training.  Some  of  the  information  presented  will  be  used  by  you  directly 
as  a  human  factors  specialist.  Other  material  will  serve  as  background 
data.  From  what  you've  studied  thus  far,  what  would  you  say  is /are  the 
Important  factor(s)  for  the  human  factors  specialist? 

(1)  The  human  factors  specialist  must  assure  himself  that  the  proper  method 
of  training  is  used  so  that  the  military  gets  the  best  trained  personnel 
possible.  Turn  to  Page  69. 

(2)  Both  of  these  answers  are  important  to  the  human  factors  specialist. 
Turn  to  Page  43. 

(3)  When  developing  a  system,  the  impact  of  personnel  selection  and 
training  in  regard  to  cost  and  time  must  be  evaluated.  Turn  to  Page  39. 


From  Page  81 


(2)  While  it's  true  that  modularization  is  an  Important  design  feature,  we 
cannot  say  that  it  will  always  be  the  most  important  feature  in  every 
design.  Return  to  Page  81. 
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From  Page  63 


(3)  Exactly  right.  Yoy  may  not  need  training  in  every  aspect  of  certain 
operations  and,  therefore,  you  do  not  want  to  simulate  the  entire  system. 
The  use  of  part-simulation  allows  you  to  accomplish  your  objectives  at  a 
lower  cost. 

To  summarize  much  of  the  content  of  the  past  two  lessons,  the 
following  selection  from  an  article  by  Alice  M.  Crawford  and  Kent  S. 
Crawford  of  the  U.  S.  Navy  Personnel  Research  and  Development  Center 
(NPRDC) ,  San  Diego  is  presented.  It  indicates  that  we  do  not  have  all  the 
answers.  Maybe  you  can  help  provide  some  of  them. 

'Individual  training  is  a  very  large  enterprise  in  the  military 
services.  Unlike  civilian  industry,  the  military  must  train  most  of  its 
personnel  in  required  skills  rather  that  select  people  who  are  already 
proficient.  Also,  given  the  trend  toward  more  sophisticated  equipment  and 
weapons  systems,  the  military  must  rely  even  more  in  the  future  op 
educational  and  training  programs  to  keep  pace  with  technological 
developments . 

'For  these  reasons,  issues  pertaining  to  the  design  and  implementation 
of  training  efforts,  which  are  of  interest  to  educators  in  all  sectors  of 
society,  are  of  particular  importance  to  the  armed  services.  Training 
technologies  must  be  effective  while  keeping  the  expenditure  of  dollars  and 
time  to  a  minimum.  Considerable  research  will  have  to  be  done  before  such 
an  objective  can  be  realized.' 

There  is  one  aspect  of  training  which  has  not  been  addressed  in  these 
lessons.  We  have  talked  about  various  design  features  and  prototypes  which 
are  used  to  assess  the  adequacy  of  the  man-machine  interface.  Think  about 
this.  How  do  we  assess  the  effectiveness  of  a  new  piece  of  equipment?  In 
Lesson  37  you  will  learn  about  test  and  evaluation,  but  for  the  moment,  you 
need  to  be  aware  that  often  it  is  necessary  to  apply  our  training  knowledge 
and  skills  to  training  test  subjects  or  participants.  In  order  to  assess 
the  man-machine  interface,  we  often  need  trained  subjects.  One  practical 
approach  to  evaluating  whether  or  not  your  test  participants  are  adequately 
trained  is  suggested  in  DI-H-7058:  give  the  trainer  a  comprehension  test 
right  after  his  training,  but  before  his  test  performance  on  the  system.  A 
more  detailed  explanation  of  test  participant  training  is  given  in  TM  29-76 
on  Pages  33-34.  Please  read  these  references  after  this  lesson  ends.  By 
the  way,  this  lesson  ends  here.  Your  next  lesson  is  coming  soon  and  is  the 
first  of  a  series  of  lessons  which  deal  with  experimental  or  test  study 
design.  Lesson  30  (Psychophysical  Methods)  begins  this  series  and  we  will 
see  you  there.  Bye. 
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HUMAN  FACTORS  ENGINEERING 


LESSON  30:  PSYCHOPHYSICAL  METHODS,  OR  DO  I  DETECT  A  SIGNAL? 


HI,  glad  to  see  you  back.  This  Is  Lesson  30  of  your  course  In  Human 
Factors  Engineering.  This  lesson  is  the  first  of  five  lessons  which  deal 
with  experimental  methodology  and  statistical  techniques.  When  I.  M. 
Eager  took  this  course  from  Professor  Ed  U.  Kator,  he  was  glad  that  the 
lessons  had  finally  come  to  this  point.  After  all,  he  had  learned  a  lot 
about  human  capabilities  and  limitations,  but,  up  to  this  point,  he  had  had 
to  rely  on  already  established  information,  such  as  that  found  in  the 
reference  documents  you  have  been  using.  Eager's  perfect  helicopter  was  so 
new  that  often  some  of  the  needed  information  was  not  contained  in  these 
reference  sources. 

Eager  (being  eager  to  be  correct)  knew  that  data  had  to  be  gathered 
and  analyzed  in  order  to  evaluate  adequately  the  design  of  his  new  machine. 
However,  he  wasn't  sure  exactly  how  to  go  about  collecting  the  necessary 
data  or,  once  collected,  how  to  analyze  it.  Now  he  would  find  out,  and  so 
will  you. 

This  lesson  is  concerned  with  methods  that  are  used  to  determine  the 
relationship  between  stimulus  intensity  and  sensation  or  the  perceived 
level  of  stimulation.  This  area  of  concern  is  called  psychophysics  and 
deals  with  the  connection  between  physical  stimuli  and  the  psychological 
responses  they  produce.  Do  you  always  detect  a  stimulus?  If  a  red  light 
appears  on  the  control  panel  of  Eager's  helicopter  to  signal  that  his 
engine  was  overheating,  what  is  the  likelihood  that  he  will  see  that  light 
in  the  excitement  and  the  workload  imposed  during  Initial  takeoff?  Can 
motivation  influence  the  way  you  respond  to  a  stimulus?  Can  you  judge  two 
equipment  stimuli  as  being  equal?  These  are  the  sorts  of  questions  that 
psychophysical  methods  may  be  used  to  answer. 

The  basic  purpose  of  the  classical  psychophysical  methods  is  to 
provide  procedures  for  determining  thresholds.  In  other  lessons  we 
discussed  the  detection  of  signals  and  the  concept  of  thresholds.  For 
example,  a  definition  of  thresholds  was  presented  in  Lesson  10,  but  we  will 
repeat  it  for  you  here.  A  threshold  is  a  stimulus  value  that  separates  the 
stimuli  which  elicit  one  response  from  those  which  elicit  another  or  no 
response.  For  example,  a  tone  of  25  Hz  presented  at  60  dB  may  be  the 
lowest  frequency  a  particular  individual  can  detect.  He  cannot  hear  a  24 
Hz  tone  at  60  dB,  but  can  hear  tones  above  25  Hz  at  60  dB.  We  would  then 
say  that  his  frequency  threshold  for  auditory  signals  presented  at  60  dB  is 
25  Hz. 


(GO  ON  TO  THE  NEXT  PAGE) 
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Now,  there  are  different  types  of  thresholds  with  which  we  are 
concerned.  Which  type  of  threshold  do  you  think  was  described  in  the 
example  just  given? 

(1)  Different  threshold.  Turn  to  Page  92. 

(2)  Absolute  threshold.  Turn  to  Page  21. 

(3)  Wedding  threshold.  Turn  to  Page  48. 

(4)  Auditory  threshold.  Turn  to  Page  8. 


From  Page  67 


(1)  Very  good.  This  is  correct.  The  average  of  all  the  adjustments  made 
by  the  subject  represents  the  stimulus  value  that  the  observer  feels  is 
equal  to  the  standard  stimulus.  This  value  is  called  the  point  of 
subjective  equality. 

Using  the  method  of  adjustment,  you  can  also  determine  the  magnitude 
of  the  constant  error  of  perception  that  is  made  in  judging  the  size  of  a 
stimulus.  This  information  is  helpful  in  the  design  of  a  variety  of 
equipment.  For  example,  if  an  operator  is  going  to  be  required  to  adjust 
controls  to  match  a  standard,  knowing  the  constant  error  for  a 
representative  group  of  operators  will  allow  you  to  'build  in'  the 
necessary  control  factor  to  compensate  for  the  expected  constant  error. 

The  third  classical  psychophysical  method  is  called  the  method  of 
constant  stimuli.  In  this  method  a  range  of  stimulus  values  is  selected 
and  a  set  of  stimulus  values  are  used  as  comparison  stimuli  (CO) .  The  COs 
are  then  presented  in  a  random  order  and  are  each  judged  against  a  standard 
value.  This  procedure  is  repeated  a  number  of  times  and  the  observer  is 
asked  to  judge  whether  the  CO  is  greater  or  less  than  the  standard.  This 
method  may  be  used  to  obtain  an  absolute  threshold  (RL),  a  difference  level 
(DL),  or  a  point  of  subjective  equality  (PSE). 

(GO  ON  TO  THE  NEXT  PAGE) 
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So,  now  you  know  what  the  three  classical  methods  of  psychophysics 
are.  It  is  also  important  for  you  to  know  some  of  the  assumptions  which 
underlie  the  use  of  these  techniques  and  which  limit  their  usefulness  to 
certain  prescribed  situations. 


In  the  classical  use  of  these  methods,  only  highly  trained  observers 
were  employed.  This  was  based  on  the  assumption  that: 


(1)  Response  biases  were  controlled  (i.e.  subjects  would  not 
anticipate  which  stimulus  value  would  be  presented  next  and  respond 
accordingly);  and 
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(2)  The  observers  were  only  influenced  by  the  stimulus  magnitude.  No 
observer-strategy  or  decision-strategy  was  felt  to  be  influencing  the 


observers'  responses. 


The  theory  of  signal  detection  was  originally  developed  to  deal  with 
problems  in  radar  and  in  radio  and  telephone  communication.  Its  major 
contribution  has  been  to  show  that  the  assumptions  underlying  classical 
psychophysics  are  false.  Signal  detection  theory  places  a  great  emphasis 
on  the  judgental  aspects,  as  well  as  the  sensory  aspects,  of  the 
psychophysical  experiment.  In  a  typical  signal  detection  experiment  an 
observer  will  be  asked  whether  or  not  he  can  detect  a  signal.  Unlike  the 
classical  methods  which  always  present  a  signal  or  stimulus,  the  signal 
detection  experiments  look  at  the  observer's  responses  both  when  a  signal 
1 8  present  and  when  it  is  not.  Figure  30.1  in  your  supplement  shows  the 
four  possible  events  in  a  signal  detection  experiment.  Please  use  that 
figure  to  answer  the  next  question. 


Which  of  the  following  situations  best  describes  a  correct  rejection? 


(1)  Indicating  that  the  patient's  heart  has  stopped  functioning  when  there 
is  a  straight-line  indicator  on  the  electric  monitor.  Turn  to  Page  49. 

(2)  Indicating  that  a  sounding  on  sonar  was  caused  by  a  moderate  size 
whale,  when  In  fact  it  was  produced  by  a  small  submarine.  Turn  to  Page  4. 

(3)  Indicating  that  an  aircraft  picked  up  on  a  dew  line  observation  is  not 
an  enemy  aircraft,  if  in  fact,  it  is  a  scheduled  TWA  Flight.  Turn  to  Page 
58. 

(4)  All  of  these  are  correct.  Turn  to  Page  84. 
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IV. 


From  Page  53 


(1)  Personnel  selection  and  training  Is  absolutely  right.  That's  why  this 
lesson  is  entitled  'training  the  right  people.'  Personnel  selection  and 
training  play  an  important  role  in  the  overall  system  design.  You  will  see 
that  a  fine  balance  must  be  maintained  between  these  two  important 
concepts . 

Several  years  ago  the  U.  S.  was  involved  in  a  race  for  the  moon.  As 
has  been  mentioned  previously,  this  race  proved  to  be  a  real  blessing  for 
human  factors  engineering.  From  what  you  know  about  selection  and 
training,  which  of  the  following  statements  would  you  deduce  to  be  true 
regarding  the  space  program? 

(1)  Since  training  was  extensive,  little  emphasis  had  to  be  placed  on 
selection.  Turn  to  Page  70. 

(2)  Since  operating  procedures  and  equipment  design  were  extremely  complex, 
great  emphasis  had  to  be  placed  on  both  selection  and  training.  Turn  to 
Page  34. 

(3)  Since  there  was  so  much  emphasis  on  selection,  the  importance  of 
training  was  minimal.  Turn  to  Page  87. 
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(2)  Totally  ignoring  the  skill  levels  of  maintainers  can  drastically  alter 
the  findings,  because  maintenance  technicians  do  not  have  equal  skill 
levels.  Return  to  Page  15. 


From  Page  58 


(4)  Do  you  really  think  this  is  the  right  answer?  It's  way  too  simplistic. 
Try  again.  Return  to  Page  58. 


(3)  Work  repair  is  someone  else's  domain,  not  yours.  As  a  human  factors 
specialist,  you  make  recommendations  for  Improvement.  You  do  not  make  the 
repairs  themselves.  Return  to  Page  37. 


From  Page  32 


(2)  Good  deduction,  and  exactly  right. 

The  space  program.  Indeed,  did  involve  a  highly  complex  and 
sophisticated  system,  so  complex  that  it  required  highly  detailed  operating 
procedures,  stringent  selection  criteria,  and  years  of  training  for  its 
personnel.  But  this  is  one  extreme.  Not  all  systems  are  so  complex.  Some 
allow  the  designers  less  rigorous  options  in  terms  of  selection  and 
training.  Recall,  however,  that  OMB  A-109  seeks  to  establish  framework  for 
all  acquistion  programs.  And  if  the  system  exceeds  $75  million  (research) 
or  $300  million  (procurement)  the  5000  series  of  DOD  directives  provide 
acquisition  policies  which  impact  on  the  designers'  options.  Recently 
TRADOC  was  advised  to  begin  stating  skill  constraints  and  training  base 
constraints  in  requirements  documents.  Therefore,  system  designers  will 
soon  have  to  understand  design  constraints  stated  in  terms  of  intelligence 
and  training  time.  Many  military  systems  fall  into  this  category.  Take, 
for  example,  a  communications  system.  The  designer  may  have  some  latitude 
as  far  as  the  complexity  of  the  system  and,  therefore,  the  level  of 
personnel  selection  criteria  and  training  requirements.  In  this  example, 
let's  assume  that  the  designer  either  selects  someone  who  'knows  the 
system'  and  needs  little  training,  or  one  who  is  trainable,  or  he  can 
simplify  operating  procedures  so  that  just  about  any  operator  can  be 
successful.  This  may  sound  quite  simple  at  first,  but  there  are  many 
questions  which  must  first  be  asked  and  answered  before  the  designer  gets 
to  this  point.  It  will  be  your  job,  as  a  human  factors  specialist,  to  pose 
these  questions  and  help  answer  them.  Before  you  can  do  this,  you'll  need 
some  more  training,  so  let's  get  some  more  details  on  personnel  selection 
and  training  requirements. 

As  a  human  factors  specialist,  you  will  be  most  concerned  with  the 
personnel  selection  and  training  process  as  it  occurs  during  design.  In 
other  words,  it's  good  for  you  to  know  the  mechanics  of  picking  the  best 
man  for  the  job.  But  more  Importantly,  you  need  to  know  how  the  equipment 
determines  who  that  man  is,  whether  or  not  he  exists,  and  whether  or  not  a 
man  can  be  trained  to  operate  and  maintain  the  equipment.  This  means  that 
if  the  human  factors  specialist  doesn't  get  involved  early  in  the  concept 
exploration  phase,  his  main  job  will  become  one  of  solving  problems.  Let's 
look  back  at  one  of  the  problems  Lt.  Eager  might  have  had  with  his 
helicopter.  If  you  recall,  one  of  the  many  'features'  of  this  helicopter 
was  its  endurance  capability.  What  if  Eager  had  only  allowed  for  one 
operator?  Can  you  imagine  how  long  It  would  take  to  find  an  operator 
capable  of  flying  for  48  hours  (selection)  or  training  one  to  do  that?  The 
point  is  that  Eager  should  have  considered  the  availability  of  capable  or 
trainable  operators. 


(GO  ON  TO  THE  NEXT  PAGE) 
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Now  that  you  know  how  not  to  do  it,  let's  examine  the  way  it's 
supposed  to  be  done. 

During  the  concept  exploration  of  development  a  series  of  top  level 
requirements  will  be  presented — the  helicopter  must  have  certain  endurance 
capabilities,  be  capable  of  supersonic  speed,  etc.  As  these  requirments 
are  converted  to  man-machine  responsibilities,  questions  will  be  asked, 
such  as  (1)  how  many  people  will  be  required  to  operate  the  system?  (2) 
what  skills  will  they  need?  and  (3)  are  skilled  people  available?  What 
happens  if  the  answer  to  this  last  question  is,  'no,  skilled  people  are  not 
available'?  What  do  you  do? 

(1)  Investigate  the  possibility  of  training.  Turn  to  Page  60. 

(2)  Reallocate  some  man  functions  to  machines.  Turn  to  Page  90. 

(3)  Both  of  these  should  be  examined.  Turn  to  Page  83. 


From  Page  55 


(4)  That  is  a  tricky  question  (or  at  least  a  tricky  answer).  It  is  rare, 
but  you  could  have  performance  validity  without  training  validity.  For 
example,  in  some  jobs,  such  as  troubleshooting,  making  decisions  between 
alternative  possibilities  of  failure  is  an  important  job  component.  It  may 
be  that  a  training  course  which  emphasized  decision-making  strategies  would 
result  in  Improved  on-the-job  troubleshooting  performance.  This  result 
could  have  occurred  even  if  the  decision-making  strategies  had  nothing  to 
do  with  actual  troubleshooting  examples.  In  this  case,  you  would  have  a 
training  component  with  little  training  validity  (no  troubleshooting 
content)  and  good  performance  validity  (improved  job  performance).  Now  you 
can  see  why  this  answer  is  not  totally  correct,  so  reread  the  selections 
and  try  again.  Return  to  Page  55. 


From  Page  83 


(1)  These  discussions  are  important,  but  what  about  the  other  techniques 
listed?  Return  to  Page  83  and  try  again. 


From  Page  87 


(1)  Safety  is  an  important  factor.  In  fact,  it  was  one  of  those  previously 
mentioned.  But  there  are  other  factors  which  also  must  be  considered. 
Return  to  Page  87 . 


From  Page  53 


(2)  You  don't  think  cost  has  been  considered  yet?  What  about  Lessons  21 
and  25?  Return  to  Page  53  for  another  answer. 
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From  Page  79 


(4)  Very  good,  you  are  correct.  These  are  all  sources  of  information  that 
may  indicate  a  potential  accident. 

The  answers  to  the  last  question  are  all  indicators  of  potential 
accidents.  Figure  27.1  in  your  supplement  shows  how  these  intermediate 
indicators  may  be  used  to  help  prevent  accidents  before  they  occur.  For 
example,  if  the  workers  complain  about  the  functioning  of  a  particular 
piece  of  equipment,  the  safety  specialists  should  investigate  to  see 
whether  the  equipment  is  about  to  malfunction  and  thereby  cause  an 
accident.  Infantrymen  who  complain  about  a  gun  jamming  are  professionals 
who  should  be  attended  to  when  their  complaints  are  made.  After  all,  who 
knows  the  equipment  better  than  its  user.  The  experienced  worker  or  user 
can  often  detect  an  out  of  control  situation  before  the  safety  inspector. 

Another  avenue  used  to  identify  critical  areas  in  the  system  is  a 
general  investigation.  All  information  about  the  system  in  question  can  be 
obtained  through  some  type  of  investigation.  Basically,  there  are  two 
types  of  safety  investigations:  preaccident  and  postaccident.  Preaccident 
investigations  can  be  initiated  by  a  variety  of  factors,  such  as  user 
complaints,  normal  rounds  of  the  safety  inspector,  etc.  Because  the 
accident  has  actually  occurred,  the  postaccident  investigator  is  usually 
able  to  isolate  the  cause  of  the  accident.  With  the  information, 
corrective  action  can  be  suggested  to  eliminate  or  reduce  the  severity  of 
future  occurrences.  Both  types  of  investigations  have  the  same  purpose: 
to  eliminate  or  extenuate  future  accidents. 

Figure  27.2  shows  you  the  various  sources  of  information  used  in  an 
accident  investigation.  In  this  figure  the  node  labeled  'accident' 
represents  the  postaccident  investigation,  while  the  node  labelled 
intermediate  indicators  impinge  upon  the  internal  or  preaccident  node, 
which,  in  turn,  leads  to  the  investigations  and  information  nodes. 

Well,  let's  review  in  the  form  of  a  question.  What  does  general 
hazard  analysis  consist  of? 

(1)  Identification,  evaluation,  corrective  action.  Turn  to  Page  48. 

(2)  Identification,  evaluation,  corrective  suggestions.  Turn  to  Page  10. 

(3)  Identification,  corrective  suggestion,  work  repair.  Turn  to  Page  33. 
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From  Page  58 


(1)  Yes  indeed,  this  answer  is  correct.  The  more  often  the  signals  are 
presented,  and  the  observer  expects  a  high  presentation  rate,  the  more 
'hits'  the  observer  will  have. 

So,  part  of  the  detection  rate  of  signals  depends  upon  the  observer's 
expectancy  about  the  rate  of  presentation.  Take  the  dew  line  observer  as 
an  example.  In  peacetime  there  are  few  aircraft,  and  the  observer  doesn’t 
really  expect  any.  In  wartime,  however,  there  is  a  different  expectancy 
concerning  the  possibility  of  enemy  aircraft.  Now,  perhaps  you  can  see  how 
the  detection  rate  will  change  as  a  function  of  the  presentation  rate. 

Another  bias  which  affects  the  detection  of  an  observer  is  the  payoff 
or  outcome  associated  with  the  observer's  responses.  In  essence,  this  is 
concerned  with  the  effect  of  motivation  on  psychophysical  judgements. 
Imagine  the  situation  in  which  you  were  to  be  paid  $10  every  time  you 
correctly  identified  the  presence  of  a  specified  signal;  there  would  be  no 
penalty  for  incorrect  responses.  How  would  this  set  of  outcomes  or  payoffs 
affect  your  responses? 

(1)  There  would  be  an  increase  in  the  number  of  misses  and  false  alarms. 
Turn  to  Page  93. 

(2)  There  would  be  an  increase  in  both  hits  and  false  alarms.  Turn  to  Page 
5. 

(3)  There  would  be  an  increase  in  the  number  of  hits,  but  a  decrease  in 
false  alarms.  Turn  to  Page  55. 

(4)  Two  of  these  answers  are  correct.  Turn  to  Page  57. 
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(3)  Exactly  right.  What  concerns  the  human  factors  specialist  most  in 
personnel  selection  and  training  is  its  impact  on  the  cost  and  the  time 
frame  required  to  implement  the  selection  or  training  program. 

In  Lesson  37,  you  will  learn  about  HFE  test  and  evaluation.  But  we 
think  it  is  a  good  idea  to  explain  that  the  HF  specialist  is  likely, 
frequently,  to  have  to  analyze  HFE  test  data  (DI-H-7058).  He  likely  will 
be  called  on  to  'recommend  solutions,'  two  of  which  are  personnel  and 
training  changes.  For  example,  in  one  tech  memo  it  was  found  that  a 
function  (fire  control)  was  allocated  to  humans.  Test  soldiers  did  poorly 
on  the  algebra  problems  required  to  perform  this  task.  So,  the  following 
alternatives  were  suggested: 

(1)  Reallocate  fire  control  function  to  machine  (and  build  a  new 
black  box  to  do  it). 

(2)  Keep  same  function  allocation,  but  provide  computational  device 
to  do  algebra. 

(3)  Get  smarter  operator. 

(4)  One  present  operator,  lots  of  extra  training. 

The  HFE  specialist  doesn't  actually  decide  which  alternative  to 
select,  but  he  is  part  of  the  design  team  which  will  have  to  make  that 
decision. 

And  so  went  Professor  Kator's  first  lesson  on  training.  Lt.  Eager  was 
briskly  jotting  down  notes  and  figuring  how  he  would  begin  the  selection 
and  training  process  for  his  super  helicopter.  Ed  U.  Kator  then  passed  out 
a  reading  assignment  for  the  next  class.  You'll  find  it  on  Pages  82-84  of 
your  supplement.  Before  you  begin  the  next  lesson,  read  through  it.  We 
think  you'll  find  it  quite  interesting.  Our  next  lesson  will  be  concerned 
with  methods  to  evaluate  the  training  process.  See  you  then! 


(1)  The  bottom  line  Is  success,  but  the  other  answers  presented  here  also 
make  some  'valid'  points.  Return  to  Page  91  and  try  again. 


From  Page  5 


(3)  Very  good,  you  are  correct.  We  must  design  equipment  with  the  lower 
difference  level  (AL)  in  mind.  We  want  all  critical  signals  to  be 
detected.  Therefore,  we  try  to  use  stimulus  values  that  are  above  the  AL. 

This  lesson  so  far  has  dealt  with  methods  for  determining  the 
acuteness  of  our  senses.  Now  we  will  present  a  few  scaling  methods  that 
are  used  to  help  us  measure  the  range  of  sensation.  For  example,  if  you 
wanted  to  present  two  sounds  and  you  wanted  your  observers  to  experience 
one  as  being  twice  as  loud  as  the  other,  would  you  make  the  physical 
intensity  (in  dB)  of  one  sound  twice  as  Intense  as  the  other  (i.e.,  40  dB 
and  80  dB)?  The  answer  is  no.  That  probably  wouldn't  result  in  a 
sensation  of  loudness  that  is  twice  that  of  the  orginal  sound.  You  would 
have  to  manipulate  frequency  as  well  as  dB. 

In  the  remainder  of  this  lesson  we  will  present  scaling  methods  which 
are  often  .called  'direct*  methods.  They  are  called  'direct'  because  they 
assume  that  the  individual  making  the  judgements  is  capable  of  describing 
his  observations  directly  on  Interval  or  ratio  scales.  Using  the  direct 
scaling  methods,  you  have  available  to  you  two  response  procedures.  One  of 
these  broad  classes  of  precedures  is  called  estimation  procedures ,  because 
the  observer  is  asked  to  estimate  directly  the  magnitude  of  the  stimulus 
presented.  The  other  class  of  procedures  Is  called  production  procedures. 
What  do  you  think  the  production  methods  require  of  the  individual? 

(1)  The  Individual  must  vary  the  stimulus  to  create  the  asked  for  value. 
Turn  to  Page  25. 

(2)  The  observer  must  judge  an  asked  for  stimulus  value.  Turn  to  Page  100. 

(3)  The  observer  must  indicate  when  his  difference  level  has  been 
established.  Turn  to  Page  95. 


(2)  Very  good,  you're  correct.  You  have  asked  the  users  to  estimate  the 
categories  that  they  feel  the  stimuli  belong  In. 


(3)  Ratio  estimation  (or  production)  -  In  this  method  the  subject's 
job  is  tb  judge  the  apparent  ratios  that  hold  between  two  or  more  stimuli 
that  are  presented.  For  example,  two  tones  might  be  presented  to  the 
observer  and  her  job  would  be  to  report  what  ratio  the  louder  tone  bears  to 
the  softer  one.  That  is,  she  might  say  the  louder  tone  would  get  a  rating 
of  90  on  a  scale  of  1  -  100,  while  the  softer  one  only  received  a  rating  of 
10.  This  allows  you,  the  human  factors  specialist,  to  know  that  there  is  a 
9  to  1  ratio  between  the  tones,  as  far  as  loudness  is  concerned,  no  matter 
what  the  intensity  distance  between  the  tones  is. 

(4)  Magnitude  estimation  (or  production)  -  typically  provides  the 
observer  with  a  ready-made  scale  to  use  (i.e.,  the  scale  of  1  -  100 
referred  to  above).  In  magnitude  estimation  procedures,  the  subjects  are 
asked  to  assign  a  numerical  property  to  the  first  stimulus  and  then  judge 
each  succeeding  stimulus  in  relation  to  the  first.  Subjects  assign 
numerical  magnitudes  to  each  stimulus  presented. 

In  magnitude  estimation  procedures,  would  it  be  possible  for  the  first 
stimulus  to  be  assigned  a  numerical  values  of  10  by  subject  1,  and  a  value 
of  250  by  subject  2,  as  well  as  a  value  of  .5  by  subject  3? 

•  (1)  Yes.  Turn  to  Page  71. 

(2)  No.  Turn  to  Page  77. 


(4)  Your  answer  is  incorrect.  One  of  the  other  choices  is  correct  though. 
Return  to  Page  2. 


(2)  Right  on,  ole  buddy.  If  we  know  the  number  and  type  of  errors  made,  we 
can  redesign  so  as  to  reduce  these  errors.  This,  of  course,  increases  the 
reliability'  of  the  human  and  the  system's  performance.  Of  course,  this 
also  points  out  the  need  for  maintainability  issues  to  be  considered  in  the 
very  earliest  stages  of  systems  acquisition. 

To  select  the  system  aspects  to  be  redesigned,  you  can  compare  two 
types  of  complexity;  task  and  hardware.  HEL  TM  22-74  (Page  11)  states  that 
task  complexity  is  found  by  dividing  the  number  of  human  performance  errors 
by  the  number  of  opportunities  for  error.  Hardware  complexity  is  found  by 
dividing  the  number  of  human  errors  by  the  number  of  parts  in  the  equipment 
group. 

Now,  if  the  equipment  is  high  on  task  complexity,  you  know  redesign  is 
a  good  idea.  The  degree  of  hardware  complexity  will  guide  you  in  deciding 
how  to  redesign. 

So,  we've  come  to  the  close  of  another  lesson.  Before  we  go,  however, 
we  must  say  one  thing  on  LSAR.  (You  remember  ...  from  your  task  analysis 
lessons  ...  there's  an  example  in  your  supplemental  section  for  Lesson  24.) 
In  the  LSAR  process,  logisticians  do  a  sort  of  task  analysis  of 
maintenance.  The  Army's  LSAR  scheme  has  recently  been  rewritten  and  is  now 
being  staffed  as  a  DOD  (Trl  service)  standard.  We  use  this  information  to 
demonstrate  the  current  emphasis  on  maintainability  and  human  performance. 

Hopefully,  it  is  now  apparent  to  you  that  Increased  maintainability 
hinges  on  a  strong  relationship  between  the  design  and  skill  level  of  the 
technician.  Less  than  a  complete  knowledge  of  either  aspect  can  do  little 
but  deflate  the  degree  and  ease  of  maintainability.  In  this  lesson  you 
have  learned  some  of  the  Important  aspects  of  designing  for 
maintainability.  As  it  should  now  be  apparent,  designing  for 
maintainability  can  have  a  substantial  Impact  on  system  performance. 
Regardless  of  how  reliable  the  system  is,  it  will  eventually  fall;  it 
requires  maintenance.  The  speed  and  ease  of  repair  will,  in  part, 
determine  how  effective  the  system  will  be  during  its  life  time.  That's 
all  for  now.  See  you  next  time  for  your  lesson  on  safety  analysis. 


(4)  Only  one  answer  Is  correct.  Go  back  to  Page  15  and  select  another 
answer . 


(2)  This  Is  a  reasonable  choice,  but  It  isn't  the  best  choice.  After 
reconsidering  the  ways  to  correct  hazard  A,  a  deferment  may  be  recommended, 
but  not  at  this  time.  Return  to  Page  93. 


I _ I 

(1)  Increasing  training  'time'  for  either  an  operator  or  maintainer  is  not 
necessarily  the  best  choice  of  answers.  In  Lessons  28  and  29  you  will 
learn  about  training.  We  guess  this  was  a  plausible  answer  since  you 
haven't  had  those  lessons  yet,  so  you  do  get  kudos  for  thinking.  However, 
you'd  better  try  another  one  to  continue  with  the  lesson.  Return  to  PagA 
52. 


From  Page  58 


V 


(3)  This  is  not  the  correct  answer.  A  low  rate  of  presentation  would  not 
increase  detection.  Lesson  16  will  help  you  determine  the  correct  answer. 
Go  back  to  Page  58. 


From  Page  22 


(2)  Exactly  right.  In  phase  one  an  inventory  of  tasks  is  developed.  From 
this  inventory  a  listing  of  tasks  which  require  training  is  produced. 

In  phase  two,  a  job  inventory  is  used  to  design  the  terminal 
objectives  and  testing  for  the  subject  course.  In  addition,  a 
determination  of  target  student  population  is  made.  Phases  three  and  four 
deal  with  the  development  and  implementation  of  the  course.  What  does  the 
final  phase  of  ISD  deal  with? 

(1)  Training  evaluation.  Turn  to  Page  69. 

(2)  Designation  of  agencies  that  will  control  the  course.  Turn  to  Page  82. 

(3)  Annual  revision  of  the  course.  Turn  to  Page  33. 

(4)  All  of  these.  Turn  to  Page  3. 


(3)  While  modular  versus  nonmodular  packaging  trade-offs  are  necessary  when 
evaluating  maintainability,  they  are  by  no  means  the  only  analysis  that 
must  be  looked  at.  Return  to  Page  68. 


(3)  Unfortunately,  a  lot  of  people  feel  this  way  and  OJT  programs  suffer 
r?  because  of  it.  Return  to  Page  9. 
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From  Page  37 


(1)  Corrective  'action'  is  not  part  of  hazard  analysis.  As  a  human  factors 
specialist,  your  job  is  to  make  recommendations  for  safety.  Return  to  Page 
37. 


From  Page  98 


(2)  You've  selected  a  partially  correct  answer.  The  tasks  themselves  might 
very  well  be  the  same,  but  the  people  and  the  organization  may  very  well  be 
different.  Return  to  Page  98. 


From  Page  30 


(3)  'Dum,  dum  de  dum. '  Nice  try.  We  appreciate  your  Interest  in  the 
Institution  of  marriage  and  in  our  attempts  at  humor.  However,  now  it  is 
time  to  attempt  a  correct  response.  Return  to  Page  30. 


From  Page  5 


(1)  Observer  bias  is  taken  into  account  in  signal  detection,  not  in  the 
classical  methods.  Return  to  Page  5. 
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(2)  This  Isn't  really  necessary.  We  know  you  have  learned  the  previous 
lessons  well  and  that  you  know  where  to  find  any  needed  data  in  the 
allltary  specifications.  Return  to  Page  6. 


From  Page  87 


(3)  Since  simulators  are  so  expensive,  savings  may  not  appear  to  be 
possible  with  the  use  of  simulators,  but  when  you  are  training  people  on 
highly  sophisticated  equipment,  cost  Is  a  factor.  It  is  not  the  only  one, 
however.  Return  to  Page  87. 


From  Page  31 


(1)  This  Is  not  correct.  In  this  situation  the  signal  (straight  line  on 
monitor)  was  present  and  the  response  was  a  'yes.'  This  answer  reflects 
the  cell  marked  'hit.'  The  technician  correctly  Identified  the  situation. 
Return  to  Page  31. 


(2)  You  would  be  correct  If  each  subject  perceived  the  stimulus  to  be  one 
half  the  first  presentation,  but  our  situation  did  not  present  it  that  way. 
Return  to  Page  71. 


From  Page  24 


(3)  Exactly  right.  The  Army  feels  that  this  systems  analysis  approach  to 
teaching  operation  and  maintenance  skills  will  work  because  of  its  broad 
perspective. 

SPAS  use  the  technical  manual  as  the  primary  tool.  However,  SPAS  are 
written  more  clearly,  accurately,  and  logically  than  most  tech  manuals.  In 
addition  to  tech  manuals,  SPAS  Include  multimedia  materials.  Similar  to 
OJT,  SPAS  have  as  their  main  advantage  the  fact  that  training  in  the  field 
reduces  the  loss  of  recall  and  retention  that  can  occur  when  training  takes 
place  at  a  formal  school.  The  training  material  used  is  designed  to 
interface  with  the  unit's  daily  mission  and  overall  training  program.  With 
this  material  the  soldier  can  continue  to  learn,  using  the  material  over 
and  over  again. 

In  addition  to  OJT-type  training  and  classroom  training,  there  is  a 
third  method  which  is  increasing  in  popularity.  This  is  computer-aided 
instruction  (CAI).  The  method  provides  individualized  instruction  using  a 
computer  to  provide  Immediate  feedback.  There  are  several  kinds  of  CAI, 
but  all  use  basically  the  same  principles.  Which  of  the  following  do  you 
think  best  describes  the  advantages  of  CAI? 

(1)  It  provides  individualized  instruction  and  immediate  feedback,  and  it 
gives  Instructors  time  to  spend  with  individuals.  Turn  to  Page  27. 

(2)  It  is  a  low— cost ,  individualized  method  of  instruction  which  provides 
immediate  feedback  to  the  student.  Turn  to  Page  15. 

(3)  It  is  a  low-cost,  individualized  method  of  instruction  which  provides 
immediate  feedback  to  the  student  in  less  time  than  traditional  methods. 
Turn  to  Page  57. 


From  Page  61 


(1)  Do  you  think  that  a  simulator  is  more  realistic  than  OJT?  If  you  do, 
you're  mistaken.  Return  to  Page  61. 


From  Page  15 


(1)  Correct.  There  are  certain  trade-offs  that  must  be  considered  when 
dealing  with  system  design  characteristics  and  maintainer  skill  level. 

In  this  lesson  we  have  addressed  the  Issues  involved  in  equipment 
design.  You  should  recall  from  your  lessons  on  systems  acquisition  and 
systems  analysis  that  the  current  focus  In  these  areas  is  on  performance 
requirements  and  specifications.  So,  too,  with  maintainability. 

The  Importance  of  assessing  the  impact  of  human  performance  on  system 
reliability  Is  quite  clear  when  you  consider  two  things. 

(1)  50-70  percent  of  all  failures  in  weapons  and  space  systems  are 
human-initiated. 

(2)  A  reliability  program  which  only  addresses  hardware  solves  less 
than  50  percent  of  the  total  reliability  problem. 

To  determine  the  reliability  of  human  performance,  you  need  a 
determination  of  the  time  required  to  perform  a  task  and  the  errors  omitted 
in  performing  it. 

TM  29-76  suggests  that'... the  full  contribution  of  human  performance 
to  operational  availability  will  not  have  been  assessed  unless  the  actual 
number  of  human  errors  (by  category)  measured  during  the  ...  test  is 
compared  with  the  opportunities  for  error,  and  that  ratio  used  either  in 
the  calculation  of  ...  system  performance  reliability  component  of  the 
operational  availability  equation.' 

If  we  use  human  reliability  Information,  we  can  then  improve  our  total 
system  reliability.  What  is  one  way  in  which  we  could  use  this 
information? 

(1)  To  increase  training  time  of  the  system  operator.  Turn  to  Page  46. 

(2)  To  selectively  redesign  so  as  to  decrease  human  performance  error. 
Turn  to  Page  44. 

(3)  To  compare  task  and  hardware  complexity.  Turn  to  Page  3. 

(4)  To  design  only  replaceable  parts  so  as  to  totally  prevent  error.  Turn 
to  Page  20. 
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Welcome  back  once  again  to  you  Human  Factors  Engineering  Course.  As 
you  recall,  Lt.  I.  M.  Eager  is  also  in  the  middle  of  a  course  in  Human 
Factors  Engineering.  His  Instructor,  Professor  Ed  U.  Kator  is  about 
halfway  through  his  lecture  series.  Thus  far,  Kator  has  discussed  the 
system  acquisition  cycle  and  how  human  factors  fits  into  it.  He  has 
examined  the  methods  of  systems  analysis  and  task  analysis  and  has  reviewed 
the  concepts  of  maintainability  and  safety.  Today  Professor  Kator  tells 
his  students  that  they  are  close  to  being  ready  to  start  practicing  what 
they  have  learned.  He  asks  them  to  think  of  a  hypothetical  system  they 
would  like  to  design  (Eager,  of  course,  thinks  about  his  super  helicopter). 
He  tells  the  students  that  although  many  design  concepts  have  been 
considered,  there  is  still  an  Important  piece  missing.  What  do  you  think 
that  one  piece  is? 

(1)  Personnel  selection  and  training.  Turn  to  Page  32. 

(2)  Cost  considerations.  Turn  to  Page  36. 

(3)  Reliability.  Turn  to  Page  22. 


From  Page  68 


(2)  It  is  true  that  trade-off  analysis  is  important  for  making  decisions 
concerning  the  value  of  repairing  versus  discarding  equipment.  However, 
other  trade-off 8  oust  be  evaluated  as  well.  Return  to  Page  68. 


From  Page  62 


(3)  One  of  these  methods  was  extensively  used  as  a  way  to  upgrade 
maintenance  skill  levels.  Return  to  Page  62. 


o 


From  Page  69 


(1)  Well  done .  This  was  a  tough  one.  Validity  actually  answers  the 
question  'How  well  does  a  predictor  predict  that  which  it  was  designed  to 
predict?'  As  we  continue,  you'll  see  how  that  relates  to  training 
programs . 

When  you  examine  how  well  a  training  program  works,  you  see  that  there 
are  really  several  different  types  of  validity.  In  general,  there  are  two: 
internal  and  external.  Internal  validity  asks  the  question  'Did  the 
training  make  a  difference  in  this  particular  setting?'  For  example, 
assume  there  is  a  course  on  aircraft  maintenance.  The  course  lasts  three 
weeks  and  a  final  exam  is  given  on  the  last  day.  If  the  course  improves  an 
individual's  performance  on  this  final  exam,  the  course  would  be  said  to 
have  Internal  validity.  The  key  here  is  that  the  validity  refers  to  a 
given  set  of  individuals  and  circumstances. 

External  validity  goes  a  step  further  and  questions  the 
generallzability  of  the  results  of  the  training  to  situations  outside  the 
training  environment.  Will  a  second  set  of  students  show  the  same  increase 
in  performance?  Can  this  training  be  used  by  the  Army  as  well  as  the  Navy? 
If  the  answers  to  these  questions  are  yes,  the  course  is  said  to  have 
external  validity. 

Irwin  Goldstein,  a  noted  industrial/organizational  psychologist,  has 
examined  In  great  detail  the  concept  of  validity  as  it  relates  to  training 
programs.  He  has  established  four  types  of  validity: 

(1)  Training  Validity 

(2)  Performance  Validity 

(3)  Intra-Organizational  Validity 

(4)  Inter-Organizational  Validity 

Training  validity  concerns  itself  with  the  performance  of  an 
individual  in  the  training  environment.  There  is  no  consideration  of  the 
transfer  of  learning  to  the  job.  So,  for  example,  a  program  which  teaches 
'management  theory'  might  have  as  its  goal  just  that — teaching  students  the 
philosophy  Involved  in  management#  If  it  is  shown  that  the  program  is 
successful  in  doing  that,  one  could  say  that  the  program  has  training 
validity.  Too  many  training  analysts,  then,  say  that  this  training  better 
prepares  a  person  for  the  job.  This  is  an  incorrect  assumption,  unless  we 
have  shown  that  the  program  has  performance  validity. 


(GO  ON  TO  THE  NEXT  PAGE) 


From  Page  54 


Performance  validity  describes  how  well  performance  has  positively 
transferred  from  the  training  program  to  the  on-the-job  environment.  Take, 
for  example,  a  course  which  teaches  aircraft  maintenance.  How  well  a 
student  performs  on  a  final  exam  could  be  a  measure  of  training  validity. 
How  well  the  student  performs  at  the  maintenance  activity  is  a  measure  of 
performance  validity. 

With  this  background  information,  what  can  be  said  about  training  and 
performance  validity? 

(1)  Both  are  examples  of  external  validity.  Turn  to  Page  77. 

(2)  If  training  validity  is  to  be  of  any  practical  value,  you  also  must 
have  performance  validity.  Turn  to  Page  91. 

(3)  Both  are  examples  of  internal  validity.  Turn  to  Page  23. 

(4)  For  performance  validity  to  be  of  any  practical  value  you  must  also 
have  training  validity.  Turn  to  Page  35. 


From  Page  38 


(3)  You're  only  half  correct.  There  would  be  an  increase  in  hits,  but  with 
no  penalty  for  an  Incorrect  response,  why  would  you  expect  a  decrease  in 
false  alarms?  Return  to  Page  38. 
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(4)  Tour  premise  Is  wrong.  What  about  a  747  cockpit  simulator  which  is 
used  to  train  pilots  how  to  react  and  make  decisions  during  emergencies? 
Return  to  Page  19. 
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From  Page  93 


(4)  Oops!  This  isn't  correct.  Only  one  answer  is  the  right  one.  Return 
to  Page  38. 


From  Page  31 


(3)  Very  good.  You  are  correct.  The  function  of  a  dew  line  observer  Is  to 
report  enemy  aircraft  (signal).  In  this  case  the  observed  plane  was  a  TWA 
flight.  Therefore,  the  signal  was  'off'  and  the  response  was  a  'no.'  A 
rejection  of  the  signal  was  correctly  given.  Keep  up  the  good  work. 

The  last  question  was  difficult,  but  by  answering  correctly,  you  have 
demonstrated  your  understanding  of  the  basic  design  of  a  signal  detection 
experiment.  The  answer  involving  the  sonarman  was  a  'miss,'  and  the  heart 
monitor  answer  was  a  'hit.'  The  correct  answer  (dew  line)  was  a  'correct 
rejection,'  so  that  leaves  only  the  cell  labeled  'false  alarm'  to  explain. 
This  Is  the  situation  In  which  an  observer  indicates  that  a  signal  is 
present  when  In  fact  it  is  not.  If  you  reverse  the  sonarman  situation  to 
one  where  there  was  no  sub,  but  the  response  was  that  there  was  a  signal, 
you  would  have  an  example  of  a  false  alarm. 

In  Lesson  16  (vigilance)  you  were  exposed  to  a  type  of  response  bias 
related  to  the  observer's  expectation  that  a  signal  would  occur.  Remember, 
in  that  lesson  you  learned  about  the  increase  in  detection  of  a  signal  as 
it  relates  to  the  rate  of  signal  presentation. 


What  was  this  relationship  between  the  rate  of  detection  and  the  rate 
of  signal  presentation? 

(1)  The  most  signals  presented  in  a  given  time  interval,  the  higher  the 
rate  of  detection.  Turn  to  Page  38. 

(2)  A  moderate  number  of  signals  presented  in  a  given  time  interval  leads 
to  the  highest  rate  of  detection.  Turn  to  Page  99. 

(3)  The  fewer  the  number  of  signals  presented  in  a  time  interval,  the 
greater  the  rate  of  detection.  Turn  to  Page  46. 


(3)  If  Che  conditions  are  similar  in  their  severity,  we  want  to  know  that* 
We  don't  want  to  readjust  our  findings.  Return  to  Page  92. 


From  Page  95 


(2)  This  probably  won't  work  too  well.  However,  if  you  were  faced  with  a 
minimum  number  of  men  and  could  not  change  the  system  design,  this  might  be 
a  last  resort.  Return  to  Page  95. 


From  Page  98 


(3)  The  task  requirements  may  be  the  same,  but  the  organizational  goals  may 
be  quite  different.  Return  to  Page  98. 
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From  Page  35 


(1)  Investigating  the  possibility  of  training  is  a  good  answer,  but  what  if 
it  turns  up  with  a  negative  result;  i.e.,  you  cannot  train  someone  to  do 
the  job?  Return  to  Page  35. 


From  Page  67 


(3)  This  isn't  the  primary  purpose  of  this  method.  In  fact,  most  of  the 
classical  methods  are  fairly  easy  to  administer  and  score.  Return  to  Page 
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From  Page  87 


(4)  Well  done.  You've  used  the  little  Information  you  have  on  simulators 
as  a  basis  for  selecting  this  answer  and  you're  right. 

To  explain  in  some  detail  the  various  reasons  for  simulator  use,  let's 
look  at  an  example.  Professor  Ed  U.  Kator  asked  his  class  to  choose  one 
and  Lt.  Eager,  of  course,  picked  his  helicopter.  Let's  follow  their  lead. 
If  you  were  responslblle  for  determining  whether  or  not  to  use  a  simulator 
to  train  operators  (pilots),  your  reasoning  might  be  as  follows  on  the 
following  subjects. 

Controlled  Reproducibility  —  If  I  use  a  simulator  I  can  control  the 
environment.  If  I  use  an  actual  helicopter,  I  have  little  control.  Some 
of  the  more  important  tasks  I  must  train  are  landing  the  helicopter,  taking 
off,  and  handling  emergencies.  With  a  simulator  I  can  present  these 
situations  over  and  over  again. 

Safety  —  Going  back  to  the  take-offs,  landings,  and  emergency 
situations,  I  really  expect  the  trainee  to  make  mistakes  at  first.  If  I 
use  an  actual  helicopter,  he  probably  will  not  have  a  chance  to  make  the 
same  mistake  twice. 

Cost  —  The  actual  helicopter  Itself  may  cost  only  $2  million  dollars. 
The  cost  of  a  simulator  may  easily  be  triple  that.  But...  what  are  the 
other  costs  Involved  in  using  the  actual  equipment?  A  mistake  by  the 
trainee  might  destroy  the  aircraft.  Having  the  student  practice  under 
emergency  situations  might  require  destruction  of  the  aircraft.  Thus,  for 
sow  sophisticated  equipment,  simulation  is  less  expensive  in  the  long  run. 

In  addition  to  these  factors,  the  trainer,  with  more  control,  can 
better  utilize  known  learning  principles.  In  this  particular  instance, 
then,  you  would  select  the  simulator  as  your  method. 

In  general,  what  would  you  say  about  your  choices  for  training 
methodology? 

(1)  Simulation  provides  a  more  realistic  replication  of  an  actual  job  than 
any  other  training  method.  Turn  to  Page  51. 

(2)  When  developing  a  training  program,  simulation  is  usually  the  preferred 
choice.  Turn  to  Page  40. 

(3)  When  cost  is  the  primary  factor,  simulation  provides  the  best  solution. 
Turn  to  Page  43. 

(4)  None  of  these.  Turn  to  Page  19. 
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From  Page  66 


I _ I 

(3)  Overall,  from  the  maintenance  point  of  view,  maintainers  at  this  level 
usually  do  not  repair  and  replace  faulty  equipment,  but  perform 
inspections,  preventive  maintenance,  and  periodic  checkout  requiring  no 
high  degree  of  skill. 

Usually,  the  least  skilled  men,  from  a  maintainability  point  of  view, 
are  those  associated  most  closely  with  equipment  operation.  Maintenance  at 
this  level  is  generally  restricted  to  monitoring  equipment  performance. 
Personnel  at  this  level  usually  do  not  repair  the  faulty  components; 
rather,  forward  them  to  a  more  skilled  level  of  maintainers.  As  is  implied 
by  the  last  statement,  maintenance  services  are  typically  stratified  into 
several  categories:  on-line  (unit)  maintenance,  intermediate 
(organizational)  maintenance,  and  depot  maintenance.  These  three 
categories  are  also  listed  in  order  of  increasing  skill  requirements  for 
technicians.  On-line  depot  level  maintenance  requires  the  lowest  level  of 
maintenance  skill.  Depot  maintenance  requires  the  most  highly  skilled  and 
specialized  technican. 

Thus,  while  some  progress  has  been  made  in  incorporating  human  factors 
engineering  into  design  for  maintainability,  much  work  still  needs  to  be 
done  at  the  point  of  Interface  between  maintenance  personnel  and  the 
equipment  being  maintained.  The  skills  of  a  maintainer  can  be  modified, 
enhanced,  or  supplemented  by  various  methods — methods  that  when  utilized 
properly  reduce  system  life-cycle  costs.  These  methods  include  such  things 
as  training,  improved  tools  and  test  equipment,  computer  programs  for 
maintenance,  maintenance  aids,  and  procedural  support,  such  as  technical 
manuals.  Before  equipment  complexity  became  overwhelming,  which  method  do 
you  think  was  considered  to  be  the  most  effective  way  of  minimizing 
downtime  (and  consequently  increasing  maintainability)? 

(1)  Maintenance  training  was  considered  the  most  effective  method  of 
minimizing  downtime.  Turn  to  Page  14. 

(2)  As  early  as  the  1940s,  computer  programs  for  maintenance  were 
considered  the  most  effective  methods  for  maintainability.  Turn  to  Page  4. 

(3)  Neither  of  these  maintenance  methods  were  considered  very  effective  in 
skill  maintenance.  Turn  to  Page  53. 


I _ I 

(1)  That's  right.  The  most  important  requirement  should  be  the  trainee's 
capability  to  perform  the  tasks  that  will  be  required  of  him.  Physical 
fidelity  is  Important  and  necessary  with  motor  tasks,  but  in  this  instance 
it  is  secondary. 

Another  aspect  of  simulators  of  importance  to  you  is  the  distinction 
between  'part-simulation'  and  'whole  simulation.'  Part-simulators 
replicate  a  critical  or  difficult  portion  of  a  task  without  attempting  to 
provide  the  complete  environment  that  is  present  in  whole-simulators.  What 
do  you  think  is  the  main  advantage  of  part-simulators? 

(1)  They  do  not  require  a  high  degree  of  functional  fidelity.  Turn  to  Page 
70. 

(2)  They  rarely  require  physical  fidelity.  Turn  to  Page  97. 

(3)  They  are  more  cost-effective  than  whole  simulators.  Turn  to  Page  28. 

(4)  All  of  the  answers  listed  here.  Turn  to  Page  45. 


From  Page  24 


(2)  Not  really.  This  method  was  developed  specifically  for  the  Army. 
Return  to  Page  24  and  try  again. 


From  Page  91 


(2)  This  is  very  true,  but  not  the  only  answer.  Return  to  Page  91. 


I _ I 

(2)  Oh,  come  now.  You  were  just  introduced  to  this  concept  as  a  definition 
of  maintainability.  We’re  sure  you  were  just  kidding.  You've  had  your 
laugh;  now  it’s  time  to  pick  the  correct  answer.  Return  to  Page  18. 


From  Page  69 


(2)  You've  made  a  common  mistake.  Reliability  reflects  the  consistency  and 
reproducibility  of  an  observation.  So  a  training  program  could  be 
consistently  unsuccessf ul-- and  therefore  reliable,  but  not  desirable. 
Return  to  Page  69. 


From  Page  5 


(2)  Remember  that  the  lower  difference  level  is  that  value  of  the  stimulus 
which  Is  just  barely  detectible.  Would  we  want  to  use  values  even  lower 
than  that?  Return  to  Page  5  and  try  again. 


(4)  You're  absolutely  correct.  Trade-offs  need  to  be  applied  to  a  variety 
of  situations.  Also,  It  is  Important  to  keep  in  mind  that  these  trade-off 
analyses  are  done  not  only  within  a  particular  area  but  also  across  areas. 

An  extremely  important  trade-off  analysis  that  must  be  undertaken 
involves  the  general  trade-offs  related  to  the  appropriate  uses  of  men  and 
machines  to  achieve  the  system  objectives.  These  decisions  are  based  on  a 
number  of  variables,  such  as  cost,  reliability,  speed,  manpower 
availability,  and  flexibility.  For  instance,  when  dealing  with  the  cost 
factors  that  affect  a  decision,  it  might  be  found  that  machine  functions 
usually  require  a  higher  initial  investment  but  are  performed  more  rapidly 
during  use.  In  general  then,  to  be  cost  effective  automated  equipment  must 
have  a  high  use  factor.  Similarly,  decisions  at  every  point  of  the 
analysis  take  into  account  the  flexibility  of  man  and  the  speed  of  response 
by  automated  equipment,  as  well  as  the  reliability  and  availability  factors 
affecting  both  sides. 

When  dealing  with  such  factors  that  affect  a  decision  on  manpower 
requirements,  operator  skill  level  is  a  very  important  source  of  input  to 
the  decision.  This  process  is  covered  thoroughly  in  Lessons  28  and  29  on 
training.  Still,  we  would  like  to  go  into  some  detail  on  how  the 
maintainability  of  a  system  is  affected  by  the  skill  levels  of  the 
maintained. 

Until  relatively  recently,  maintainability  designers  were  primarily 
concerned  with  the  efficient  accomplishment  of  the  electronic  task  on 
function,  and  very  little  attention  was  given  to  the  malntalner's 
capabilities  and  limitations.  Only  recently  has  the  training  of 
maintenance  personnel  been  considered  to  have  a  large  influence  on  design 
considerations.  Equipment  that  requires  maintenance  skill  levels  higher 
than  those  provided  by  training,  or  beyond  what  can  possibly  be  expected, 
cannot  be  maintained  successfully.  Thus,  if  the  needed  skills  greatly 
exceed  what  is  available,  the  equipment  becomes  a  liability  rather  than  an 


(GO  ON  TO  THE  NEXT  PAGE) 


asset.  With  all  of  this  Is  mind,  where  would  you  expect  to  use  the  least 
skilled  maintenance  personnel? 

(1)  The  best  skilled  maintenance  personnel  are  usually  paired  with 
maintenance  experts  in  a  rework  type  facility.  Turn  to  Page  83. 

(2)  In  places  that  are  most  completely  removed  from  equipment  operation. 
Turn  to  Page  97. 

(3)  Usually  the  least  skilled  maintenance  personnel  are  those  associated 
most  closely  (in  terms  of  physical  proximity)  to  equipment  operation.  Turn 
to  Page  62. 

(4)  These  people  are  typically  assigned  to  nonpower  or  automated  equipment 
so  as  to  reduce  the  potential  for  errors.  Turn  to  Page  33. 


From  Page  21 
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When  using  this  method  to  determine  a  DL,  the  observer  is  presented 
with  a  comparison  stimulus  and  a  standard  stimulus  on  each  trial.  His  task 
is  to  indicate  whether  the  comparison  stimulus  is  equal  to,  greater  than, 
or  less  than  the  standard.  In  this  case,  two  thresholds  are  established: 
the  upper  one,  which  Indicates  a  change  from  an  equal  response  to  a 
positive  one,  and  the  lower  threshold,  which  indicates  a  change  from  an 
equal  response  to  a  negative  one.  The  DL  is  established  as  the  average  of 
these  two  thresholds. 

You  probably  have  had  experience  with  a  variation  of  the  classical 
method  of  limits.  If  ycu've  ever  had  your  hearing  tested,  you  probably 
used  an  audiometer.  This  Instrument  presents  a  tone  which  grows  louder  and 
louder  (ascending  series)  until  you  push  a  button  or  indicate  in  some 
manner  that  you  can  hear  the  tone.  Then  it  grows  fainter  (descending 
series),  until  you  can  no  longer  hear  it.  On  the  next  trial,  a  different 
frequency  may  be  presented  in  a  similar  manner.  Anyway,  you  get  the  idea. 

The  second  classical  method  is  called  the  method  of  adjustment.  In 
this  method  an  observer  adjusts  a  stimulus  until  he  perceives  it  to  be 
equal  to  a  standard  stimulus  that  has  been  presented  by  the  experimenter. 
For  example,  the  experimenter  may  present  a  bright  light  of  a  certain 
luminaire  level  and  ask  the  observer  to  adjust  another  light  source  until 
he  feels  it  is  equal  in  brightness  to  the  first  light. 

The  main  purpose  of  this  method  is  not  to  determine  a  threshold, 
although  it  can  be  used  to  determine  AL  or  DL.  Given  the  explanation 
above,  which  of  the  following  do  you  think  is  the  primary  purpose  of  using 
the  method  of  adjustment? 

(1)  To  establish  a  point  of  subjective  equality  (FSE).  Turn  to  Page  30. 

(2)  To  establish  a  point  of  objective  equality  (POE).  Turn  to  Page  23. 

(3)  The  method  of  adjustment  is  used  only  when  it  is  impossible  to  use  one 
of  the  other  methods.  Turn  to  Page  60. 

(4)  None  of  these  is  a  correct  answer.  Turn  to  Page  71. 


(4)  You're  quite  right.  All  of  these  features  should  be  carefully 
evaluated  from  a  systems  standpoint,  and  those  features  which  are  most 
important  in  a  particular  system  depends  on  the  unique  functions  of  that 
system.  Thus,  the  importance  of  a  feature  will  vary  from  system  to  system. 

As  we  discussed  earlier,  systems  analysis  involves  the  allocation  or 
organization  of  functions  to  provide  a  structure  for  making  decisions 
concerning  man-machine  trade-offs.  The  increasing  complexities  and 
subsequent  high  costs  of  equipment  design  and  maintenance  have  increased 
the  importance  of  making  correct  design  decisions;  one  can  no  longer  depend 
on  intuition  or  guesswork.  Trade-off  analysis  is  a  decision-making 
technique  that  can  help  reduce  the  possibility  of  making  a  poor  decision. 
Thus,  trade-off  analyses  are  required  to  provide  evidence  that  all  related 
factors  have  been  systematically  evaluated  prior  to  making  a  decision. 

Taking  into  account  the  information  that  has  been  presented  over  the 
last  several  lessons,  as  well  as  what  has  been  discussed  thus  far  in  this 
lesson,  in  which  of  the  following  areas  should  trade-off  methodologies  be 
applied  in  designing  for  maintainability? 

(1)  Trade-off  between  reliability  and  maintainability.  Turn  to  Page  3. 

(2)  Trade-off  between  repair  and  replacement  of  equipment.  Turn  to  Page 
53. 

(3)  Trade-off  between  modular  versus  nonmodular  packaging.  Turn  to  Page 
47. 

(4)  All  of  these  areas  must  be  evaluated  in  terms  of  the  trade-offs 
involved.  Turn  to  Page  65. 


Prom  Page  47 


(1)  That's  It.  Evaluation  and  revision  are  the  key  elements  of  this  final 
phase.  This  lesson  will  deal  with  the  overall  evaluation  process  and  its 
importance  to  you  as  a  human  factors  specialist. 

So,  you  see  that  an  important  part  of  training  is  making  sure  it 
works.  Obviously,  a  training  program  is  of  limited  use  if  it  fails  to 
achieve  its  goals.  What  do  you  think  is  the  technical  term  used  to  measure 
this  success? 

(1)  Validity.  Turn  to  Page  54. 

(2)  Reliability.  Turn  to  Page  64. 

(3)  Probability.  Turn  to  Page  78. 

(4)  All  of  these  answers.  Turn  to  Page  15. 


From  Page  79 


(1)  This  is  only  one  source  of  potential  accident  information;  there  are 
others.  Return  to  Page  79. 


From  Page  27 


(1)  The  human  factors  specialist  is  concerned  with  training,  but  not  at 
this  level.  This  is  the  job  of  a  training  specialist.  Return  to  Pagt  27. 
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(2)  While  the  military  has  as  one  of  its  purposes  defense  through  offense, 
it  has  also  been  very  strict  in  its  safety  regulations.  Return  to  Page  16. 


! 

From  Page  32 


(1)  In  rare  instances,  if  enough  training  takes  place, 
write  off  the  need  for  selection  (assuming  you  can 
training),  but  that  wasn't  the  case  in  the  space  program. 
32. 


you  can  all  but 
afford  all  the 
Return  to  Page 


From  Page  63 


(1)  C'mon!  What  good  is  a  simulator  (part  or  whole)  that  does  not  have  a 
high  degree  of  functional  fidelity?  Return  to  Page  63. 


(4)  Exactly  right.  All  of  these  are  ways  to  determine  skills  required  for 
a  specific  job  and  they  are  all  ways  of  studying  the  human  performance 
requirements. 

Now,  let' 8  review  the  overall  approach  to  personnel  and  training 
requirements  in  terms  of  the  following  example: 

(1)  An  operational  requirement  exists  (e.g.  protect  the  coast  of  the 
U.  S.  from  SNX-4  missile  attack); 

(2)  A  system  made  up  of  several  elements  or  subsystems  is  conceived 
to  meet  the  objectives  of  the  operational  requirement; 

(3)  The  system  is  operationally  defined  in  terms  of  the  functions  it 
performs.  These,  in  turn,  are  accomplished  by  carrying  out  a  series  of 
tasks  which  are  performed  at  various  positions  in  the  system  by  either  man 
or  machine; 

(4)  Certain  skills  are  required  of  personnel  to  perform  their 
assigned  tasks; 

(5)  Numbers  and  time  frame  requirements  for  personnel  are  determined. 

With  this  overall  viewpoint  of  selection  and  training  process  in  mind, 
let's  return  to  the  concept  exploration  stage  of  systems  acquisition.  At 
this  stage  a  series  of  alternate  approaches  to  meeting  the  mission 
requirements  exists.  How  does  the  human  factors  specialist  assist  in 
determining  the  best  approach?  -  Through  a  series  of  analyses. 
Specifically,  through  organizational,  job,  and  person  analyses. 

First,  let's  examine  organizational  analysis.  The  goals  of  the 
military  organization  a  human  factors  specialist  is  involved  with  are 
usually  fairly  obvious;  namely,  to  be  prepared  to  defend  the  nation  from 
'foes  both  foreign  and  domestic.'  However,  the  resources  which  a  service 
has  available  to  meet  these  goals  require  some  analysis.  This  should 
include  an  analysis  of  available  equipment  (Do  you  already  have  a  system 
which  meets  the  requirements?),  the  organization's  financial  resources, 
and,  perhaps  most  importantly,  manpower  availability. 

The  second  in  this  series  of  analyses  is  the  job  analysis.  Uhat  do 
you  think  is  the  main  purpose  of  a  job  analysis? 

(1)  To  describe  the  important  aspects  of  a  job  which  are  critical  for  job 
success.  Turn  to  Page  94. 

(2)  To  determine  the  cost  of  performing  a  given  job.  Turn  to  Page  85. 

(3)  To  determine  whether  or  not  a  given  job  can  be  performed.  Turn  to  Page 


From  Page  72 


(3)  Absolutely  correct.  You  are  doing  very  well. 

You  would  recommend  a  reanalysis  for  hazard  A.  There  may  be  a  variety 
of  ways  to  abate  the  hazard.  Each  of  these  methods  would  have  a  different 
level  of  expense  associated  with  it.  Each  also  would  probably  have  a 
different  level  of  effectiveness  associated  with  it  as  well.  By 
reanalyzing,  you  could  arrive  at  a  solution  that  was  both  cost-ef fective 
and  hazard -reducing.  This  analysis  is  where  the  experience  of  the  safety 
officer  comes  into  play. 

DI-H-7058,  which  is  the  DID  for  human  engineering  test  report,  has  a 
block  (block  10)  on  how  to  prepare  instructions.  Paragraph  7  is  denoted  to 
the  description  of  observed  safety  hazards.  This  section  contains 
instructions  on  the  data  necessary  for  the  evaluation  phase  of  hazard 
analysis.  DI-H-7058  is  in  your  supplement  on  Page  129. 

Essentially,  this  section  of  the  DID  requires: 

(1)  A  narrative  description  of  each  safety  hazard  and  any  appropriate 
photographs.  In  addition,  each  hazard  is  analyzed  as  to  the  frequency  with 
which  it  was  encountered  by  the  test  participants. 

(2)  The  severity  of  each  hazard  is  determined  and  the  consequences  of 
each  hazard  are  delineated. 

(3)  The  actions  recommended  are  stated  in  terms  of  design,  task 
alternatives,  and/or  personnel.  It  may  reduce  the  hazard  and  be  more  cost 
effective  to  allocate  functions  differently  than  current  situations.  Or, 
the  hazard  may  only  be  eliminated  or  reduced  by  equipment  redesign. 
Remember,  the  name  of  the  game  is  safety,  but  the  rules  require  system 
effectiveness  within  the  established  cost  parameters. 

In  this  lesson  it  may  appear  that  the  safety  officer,  as  an 
individual,  is  making  all  the  decisions.  This  is  not  the  case  in  most 
situations.  The  safety  officer  is  part  of  a  team.  You,  as  a  human  factors 
specialist,  would  probably  be  a  part  of  this  team. 

Now,  there  are  dozens  of  ways  to  perform  a  hazard  analysis  and  even 
more  methods  for  determining  cost  effectiveness.  We  don't  expect  you  to  be 
proficient  in  hazard  analysis  from  this  lesson.  However,  it  is  helpful  to 
you  as  a  human  factors  specialist  to  be  aware  of  all  facets  of  the  system 
in  which  you  are  involved. 


(GO  ON  TO  THE  NEXT  PAGE) 
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For  example,  an  R  and  D  team  was  called  In  to  evaluate  a  new 
structural  arrangement  to  be  placed  on  the  flight  deck  of  an  aircraft 
carrier.  The  purpose  of  the  structure  was  to  house  men  who  were  Involved 
In  nuclear  missile  attack  capabilities  in  the  event  that  the  system  had  to 
be  operated.  Once  the  men  had  entered  the  structure,  it  would  be  sealed 
and  could  only  be  opened  by  a  maneuver  from  the  bridge.  The  men  inside  the 
structure  could  not  open  the  seal. 

Only  the  human  factors  specialists  on  the  R  and  0  team  saw  a  major 
flaw  In  that  system.  All  the  other  members  of  the  team  were  concerned  with 
the  hardware  operations  or  the  manpower  planning  scheme.  Uhat  major  flaw 
do  you  think  the  human  factors  engineers  found? 

(1)  There  was  a  fire  hazard  on  the  structure  which  all  others  missed.  Turn 
to  Page  90. 

(2)  If  the  bridge  were  knocked  out,  the  men  could  not  escape  the  structure. 
Turn  to  Page  96. 

(3)  There  was  no  toilet.  Turn  to  Page  23. 

(4)  The  personnel  were  not  thoroughly  trained.  Turn  to  Page  100. 


From  Page  87 


(2)  Controlled  reproducibility  is,  indeed,  a  key  reason  for  the  selection 
of  a  simulator,  but  there  are  other  considerations  also.  Return  to  Page  87 
and  try  again. 


From  Page  77 


(3)  While  It's  true  that  a  perfect  design  might  allow  anyone  to  maintain 
the  equipment,  there  are  definite  trade-offs  between  equipment 
sophistication  and  simplicity  of  design.  Anyway,  regardless  of  the 
ramifications,  each  of  diagnosis  Is  not  necessarily  related  to  designing 
for  accessibility.  Return  to  Page  77. 


From  Page  9 


(4)  Look  again.  You're  making  some  incorrect  assumptions.  For  example,  do 
you  really  think  that  OJT  programs  require  less  effort  than  others?  Return 
to  Page  9. 


From  Page  25 


>T« 


(1)  Good  show,  you're  absolutely  right.  Reliability  refers  to  the  ability 
of  the  system  to  remain  free  from  failure. 

Before  we  move  on  to  other  aspects  Important  when  designing  for 
maintainability,  let's  stop  and  spend  a  little  time  discussing  some 
important  ways  in  which  we  can  quantitatively  measure  maintainability  and 
reliability.  Notice  that  in  defining  these  two  terms,  a  specific  time 
element  is  Incorporated  as  a  part  of  each  definition,  thus  emphasizing  the 
need  for  an  assurance  that  'maintenance  time'  will  not  exceed  a  specified 
value.  Therefore,  both  of  these  concepts,  when  quantitatively  measured, 
become  the  system's  predictors  of  'operational  availability.'  An  oft~used 
operational  availability  measure  is  mean  time  to  repair  (MTTR).  The  MTTR 
is  the  average  time  it  takes  a  technican  of  a  specified  skill  level  to 
locate  a  problem,  repair  or  replace  the  unit,  and  verify  that  the 
correction  has  been  made. 

In  addition  to  this  measure  of  maintenance  a  quantitative  measure  of 
reliability  is  also  used  frequently  ■”  mean  time  between  failures  (MTBP). 
This  operational  availability  measure  focuses  on  the  total  system's 
operating  time  divided  by  the  number  of  system  failures  during  that  time 
period.  These  quantitative  measures  are  an  Important  source  of  input 
during  the  design  cycle.  However,  neither  of  these  operational 
availability  measures  Incorporates  the  human  performance  involved  in 
maintainability.  By  looking  at  MTTR  or  MTBF,  valuable  information  can  be 
gained.  Unfortunately,  the  full  contribution  of  human  performance  to 
operational  availability  will  not  have  been  assessed  unless,  for  example, 
the  actual  number  of  human  errors  measured  at  any  period  of  time  is 
Included  in  a  measure  of  operational  availability. 

In  the  past,  measures  of  human  performance  reliability  have  been  a 
'sore  thumb'  to  maintainability  designers.  This  is  because  the  'human' 
part  of  the  system  has  not  traditionally  been  incorporated  into  the  design 
during  early  stages  of  the  materiel  life-cycle.  Then,  when  man  was 
Introduced  as  part  of  the  system,  he  no  longer  was  an  asset  to  the  system 
(often  times  he  no  longer  'fit,'  if  you'll  remember  our  earlier  lessons  on 
anthropometry).  However,  with  the  aid  of  human  factors  specialists,  we 
have  begun  to  pull  out  of  the  'dark  ages  of  designing  for  maintainability,' 
as  will  become  apparent  to  you  by  the  time  you  finish  this  lesson. 

Now  that  you  are  familiar  with  the  concepts  of  maintainability, 
reliability,  and  maintenance,  It's  time  to  move  on  to  other  important 
aspects  of  maintainability.  We  will  begin  with  a  discussion  of  some  design 
factors  that  have  been  found  to  enhance  systems  maintainability. 
Throughout  this  discussion  you  should  keep  in  mind  that  these  features  are 
generally  applicable,  but  that  they  should  not  be  applied  blindly.  Strict 
adherence  to  the  features  and  suggestions  discussed  in  this  section  is  to 
be  avoided;  their  generality  across  all  situations  is  unlikely. 


(GO  ON  TO  THE  NEXT  PAGE) 


From  Page  76 


One  important  design  feature  is  that  of  accessibility, 
maintainability,  what  do  you  think  accessibility  refers  to? 


In  terms  of 


(1)  Simplification  of  design  so  as  to  allow  the  maintainer  to  easily  assess 
the  problem.  Turn  to  Page  84. 

(2)  The  relative  ease  with  which  a  component  of  an  assembly  can  be 
approached  to  be  repaired,  replaced,  or  serviced.  Turn  to  Page  88. 

(3)  Designing  equipment  to  enable  even  the  lowest  level  maintainer  to 
repair  any  problems.  Turn  to  Page  75. 


From  Page  55 


(1)  You'd  better  review  the  definitions  again.  External  validity  concerns 
Itself  with  generalizing  outside  the  training  environment.  This  is. 
Indeed,  the  concern  of  performance  validity,  but  not  training  validity. 
Return  to  Page  55. 


From  Page  16 


(1)  Very  good,  the  military  has  been  forerunner  In  safety  design. 

The  various  military  services  have  developed  very  sophisticated  safety 
programs.  They  are  aware  of  the  value  of  men  and  equipment.  The  military 
organizations  do  have  an  advantage  over  private  sector  industries  in  that 
they  have  a  great  deal  more  control  over  their  personnel  then  the  private 
sector  does. 

So,  now  you  know  that  one  general  way  of  identifying  potential 
accidents  is  through  historical  records.  Can  you  think  of  specific  types 
of  information  to  be  found  in  company  records  that  might  also  indicate  the 
source  of  potential  accidents?  Give  it  a  shot,  which  of  the  following 
would  show  a  possible  hazard? 

(1)  Reports  of  unusual  delays  in  the  normal  routine.  Turn  to  Page  69. 

(2)  Reports  of  damaged  goods.  Turn  to  Page  60. 

(3)  Reports  of  employee  complaints.  Turn  to  Page  SO. 

(4)  All  of  these  can  be  used  as  sources  of  potential  accident  information. 
Turn  to  Page  37. 


(3)  Right.  Each  hazard  discovered  should  first  be  grouped  according  to  its 
severity  ranking.  Return  to  Page  11. 
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(3)  You're  joking,  right?  If  you'll  stop  and  think  for  a  minute,  we  hope 
you'll  remember  this  sentence  being  presented  to  you  as  a  definition  of 
maintenance.  Maybe  it  would  be  wise  if  you  went  back  and  reread  part  of 
this  lesson  again.  Return  to  Page  18. 


From  Page  24 


(1)  Most  people  are  more  enthusiastic,  and  therefore,  more  motivated  by 
'hands  on'  experience  rather  than  classroom  training.  SPAS  give  an 
alternative  to  that  type  of  formal  training.  Return  to  Page  24. 


From  Page  88 


(4)  You're  absolutely  right.  These  locational  recommendations  are  all  good 
guidelines  to  follow  when  designing  for  maintainability. 

Okay,  so  far  we  have  discussed  two  important  maintainability  design 
features;  namely;  accessibility  and  ease  of  replacement.  In  addition, 
several  other  factors  should  be  discussed  that  are  important  when  designing 
for  maintainability.  We  have  alluded  to  unit  packaging  placement  in  the 
preceding  discussion,  and  it  needs  to  be  pointed  out  that  'unit  packaging' 
is  one  such  suggested  design  feature.  Packaging  equipment  into  separate, 
replaceable  units  can  enhance  maintainability  by  facilitating 
troubleshooting,  reducing  handling  problems,  and  making  it  easier  to 
replace  malfunctioning  parts.  Thus,  modular  or  unit  packaging  is 
recommended  whenever  feasible.  Along  these  same  lines,  replaceable  modules 
should  be  independent  and  interchangeable,  if  at  all  possible.  For 
instance,  you  might  have  had  the  unfortunate  experience  of  needing  to 
replace  the  spark  plugs  on  your  car,  only  to  find  that  the  air  conditioning 
system  had  to  be  either  partially  or  totally  disassembled  beforehand. 
Obviously,  such  poor  design  features  have  a  negative  impact  on 
maintainability. 

When  designing  for  maintainability,  it  is  also  a  good  idea  to  include 
self-checking  features  or  test  points  that  can  be  tested  by  auxiliary  test 
equipment.  To  be  most  useful,  test  points  (or  spots  at  which  a  technician 
can  trouble-shoot  various  portions  of  a  system)  must  be  in  the  proper 
functional  location.  Therefore,  the  test  points  should  be  tied  into 
circuits  where  the  technician  needs  to  sample  signals,  as  well  as  in  places 
where  the  technician  can  reach  them  easily. 

Now,  we  have  just  discussed  a  number  of  design  features  important  for 
effective  maintainability.  Which  one  of  these  features  would  you  consider 
to  be  most  important  for  designing  for  maintainability? 

(1)  Ease  of  access  to  test  points  and  internal  parts  of  the  equipment. 
Turn  to  Page  22. 

(2)  Modular  or  unit  packaging.  Turn  to  Page  27. 

(3)  Replaceable  modules  or  units  that  are  independent  and  interchangeable. 
Turn  to  Page  7. 

(4)  All  of  these  features  should  be  carefully  evaluated  from  the  overall 
system  point  of  view.  Turn  to  Page  68. 
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From  Page  13 


(1)  We've  done  this  too.  We  keep  saying  it  is  the  rug's  fault  and  not  our 
clumsiness.  Do  you?  Anyway,  this  isn't  the  only  source  of  hazard 
recognition.  Return  to  Page  13. 


From  Page  47 


(2)  The  designation  of  agencies  responsible  for  controlling  the  course  will 
have  been  made  much  earlier  than  this.  Return  to  Page  47. 


From  Page  7 1 


(3)  You've  missed  the  point  completely.  Remember,  assigned  values  are 
based  on  relationships  of  perceived  stimulation.  Return  to  Page  71. 
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••  (3)  Exactly  right.  Which  of  the  two  you  choose  depends  on  several  factors. 

Now  is  the  time  for  a  trade-off  analysis. 


You  have  determined  the  skills  required  to  operate  and  maintain  your 
system,  you  have  analyzed  your  available  skills  and  you  discover  a  gap 
which  needs  to  be  bridged.  If  that  gap  is  not  too  wide,  personnel 
selection  and  training  come  into  play.  Before  we  go  any  further  though, 
answer  the  following  question.  How  did  you  determine  the  skills  required 
for  the  system? 

(1)  By  talking  with  operators  of  similar  equipment.  Turn  to  Page  36. 

(2)  Job  analysis.  Turn  to  Page  56. 

(3)  By  taking  with  the  equipment  designers.  Turn  to  Page  16. 

(4)  All  of  these.  Turn  to  Page  72. 


(1)  While  It  would  probably  be  a  good  OJT  technique  to  pair  skilled  and 
unskilled  personnel,  this  answer  really  doesn't  apply  to  the  question. 
Return  to  Page  58. 


(1)  While  it  would  probably  be  a  good  OJT  technique  to  pair  skilled  and  •'-'•VVj 

unskilled  personnel,  this  answer  doesn't  apply  to  the  question.  Return  to 
Page  66 . 
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(1)  This  Is  not  a  correct  answer.  An  ability  to  assess  the  problem  is 
important  in  maintainability,  but  this  is  not  a  part  of  designing  for 
accessibility.  Return  to  Page  77. 
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(1)  This  Is  not  the  best  possible  answer.  Still,  you  are  correct  In 
assuming  that  this  answer  is  Important  in  maintainability  design.  Return 
to  Page  88. 


From  Page  72 


(2)  The  cost  of  performing  a  job  is  an  important  factor,  but  it  is  not  the 
responsibility  of  job  analysis  to  determine  the  cost.  Return  to  Page  72. 


From  Page  71 


(1)  You've  missed  the  point  completely.  Remember,  assigned  values  are 
based  on  relationships  of  perceived  values.  Return  to  Page  71. 


(1)  Exactly  right*  It  is  very  dangerous  to  attempt  to  generalize  the 
training  results  in  one  organization  to  trainees  in  another  without  a  needs 
assessment . 

This  generalization  from  one  organization  to  another  is  what  Goldstein 
calls  inter-organizational  validity.  The  point  made  here,  as  was  made 
above,  is:  Do  not  borrow  the  results  from  one  level  of  validation  and 
apply  it  to  another.  Do  not  assume  you  have  an  intra-organizatlonally 
valid  training  program  when  all  you've  conducted  is  performance 
validation. 

Let' 8  pause  for  a  moment  and  look  at  a  question  which  should  be 
foremost  in  our  minds.  That  question  is:  How  does  this  affect  me  as  a 
human  factors  specialist?  The  answer  to  this  can  be  found  in  Army 
Regulation  No.  602-1,  'Human  Factors  Engineering  Program.'  Summarizations 
of  some  HFE  responsibilities  in  this  document  are  the  following: 

(1)  Coordination  with  other  agencies  in  determining  the  needs  for, 
and  then  developing  and  evaluating  job  procedures,  performance  aids, 
training  devices  and  aids,  equipment,  and  technical  publications. 

(2)  Assessing  the  training  burden  which  competing  material  design 
concepts  may  impose  on  the  military. 

(3)  Developing  the  information  needed  for  new  or  revised  training 
plans,  courses,  or  programs  of  instruction  as  required  by  new  or  modified 
material,  doctrine,  or  organization. 

The  information  presented  thus  far  will  assist  you  in  accomplishing 
these  missions.  The  preceding  lesson  showed  you  how  to  assess  training 
needs.  This  lesson  will  guide  (or  assist)  you  when  training  evaluation  is 
required.  Be  aware,  however,  that  training  evaluation  is  highly  technical 
and  requires  a  professional  to  conduct  it.  The  remainder  of  this  lesson 
will  cover  training  devices  and  your  role  in  their  development. 

In  recent  years  the  Department  of  Defense  has  placed  Increased 
emphasis  on  using  sophisticated  simulation  training  devices  rather  than 
actual  equipment  for  the  training  of  complex  tasks.  Ideally,  the 
development  of  these  training  devices  is  guided  by  training  requirements, 
which  are  in  turn  determined  by  task  analysis.  Let's  take  a  look  at 
simulation — its  theory  and  its  application. 

(GO  ON  TO  THE  NEXT  PAGE) 


I _ I 

As  you  may  recall  from  the  last  lesson,  simulation  training  is  merely 
one  of  several  different  methods.  Why  is  it  so  important?  Its  importance 
lies  in  the  fact  that  for  complex  tasks  it  is  often  the  'best'  method. 
Think  about  what  you  know  of  simulators  and  answer  the  following  question. 
What  do  you  think  are  the  major  reasons  for  selecting  simulation  as  a 
training  method? 

(1)  Safety  considerations.  Turn  to  Page  36. 

(2)  Controlled  reproducibility.  Turn  to  Page  74. 

(3)  Cost.  Turn  to  Page  49. 

(4)  All  of  the  answers  listed  here.  Turn  to  Page  61. 


From  Page  16 


(3)  In  between  what?  The  question  requires  an  answer  in  one  direction  or 
the  other,  not  both.  In  this  course  you  can't  have  your  cake  and  eat  it 
too.  Return  to  Page  16. 


From  Page  32 


(3)  You  obviously  read  the  news  when  they  advertised  for  astronauts,  but 
you  didn't  when  they  told  of  the  years  of  training  required.  Return  to 
Page  32  and  try  again. 


I _ I 

(2)  Good  show,  you're  absolutely  right*  It  is  fundamentally  important  that 
a  malntalner  be  able  to  gain  access  to  the  equipment  he/she  is  to  maintain. 
Thus,  one  principle  of  designing  for  maintainability  is  to  design  for 
accessibility. 

Poor  accessibility  to  routine  service  points  and  parts  of  equipment 
reduces  the  efficiency  of  the  maintenance  operation,  while  increasing  the 
time  required  for  maintenance.  If  service  points  are  difficult  to  reach  or 
use,  routine  maintenance  tasks  might  not  be  performed  as  required,  or  they 
might  not  be  performed  at  all.  Thus,  accessibility  is  an  Important  feature 
that  should  be  taken  into  consideration  when  designing  for  maintainability. 

Another  Important  factor  to  be  considered  in  designing  for 
maintainability  is  ease  of  replacement.  Once  a  malfunction  has  been 
diagnosed,  the  next  logical  step  is  to  correct  that  problem.  The  most 
obvious  way  to  minimize  the  system's  'downtime'  (the  total  time  a  system  is 
down  (inoperative)  for  active  maintenance)  is  to  design  the  system  so  as  to 
provide  for  easy  replacement  of  malfunctioning  parts.  There  are  a  number 
of  design  recommendations  that  should  be  used  as  'rules  of  thumb'  when 
planning  for  the  replacement  of  units.  Which  of  the  following 
recommendations  would  you  consider  to  be  Important  for  ease  of 
replacability? 

(1)  Locate  each  unit  in  the  equipment  in  such  a  way  that  no  other  unit  or 
piece  of  equipment  has  to  be  removed  to  get  to  the  unit.  Turn  to  Page  85. 

(2)  If  it  is  necessary  to  put  one  unit  behind  another,  the  unit  requiring 
less  frequent  attention  should  be  placed  behind  the  one  requiring  more 
frequent  attention.  Turn  to  Page  13. 

(3)  Do  not  put  a  unit  in  a  recess,  or  behind  or  under  other  struct  ral 
units,  floor  boards,  operators'  seats,  hoses,  pipes,  or  other  equipment 
parts  that  are  difficult  to  remove.  Turn  to  Page  98. 

(4)  All  of  these  answers  are  important  'ease  of  replacement' 
recommendations.  Turn  to  Page  81. 


wrong  answer.  Nothing  was  said  about  a  fire  hazard. 


Return  to  Page  74. 


From  Page  35 


(2)  Reallocating  functions  may  be  the  answer,  but  it  might  not  be  the  best. 
Return  to  Page  35. 


From  Page  25 


(4)  This  example  wasn't  a  signal  detection  experiment.  Try  again.  Return 
to  Page  25. 
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From  Page  55 


(2)  Exactly  right.  We  doubt  that  you'll  find  an  organization  that  is 
interested  primarily  in  training  validity  (with  the  exception  of  a  school). 
Most  organizations  want  an  indication  that  job  performance  will  improve. 

As  you  recall,  before  developing  any  training  program,  a  series  of 
analyses  must  take  place.  This  is  often  referred  to  as  a  needs  assessment. 
You  examine  what  tasks  are  required,  the  importance  and  frequency  of  each 
task  in  the  ultimate  setting  (whether  that  be  a  final  exam  or  on  the  job), 
and  how  difficult  it  is  to  learn  how  to  do  a  given  task.  In  addition,  you 
need  to  assess  the  characteristics  of  the  trainee  population.  The  task 
Inventory  that  is  used  for  other  purposes,  such  as  design  and  workload 
analysis,  is  the  same  one  that  you  would  use  for  training  analysis. 

Before  you  can  hope  to  establish  acceptable  training  validity,  both 
the  task  requirements  and  the  trainee  population  characteristics  need  to  be 
assessed.  Using  only  these  two  analyses  would  prove  insufficient  to 
establish  performance  validity.  Why  do  you  think  we  say  this? 

(1)  Because  history  has  shown  that  training  programs  which  ignore 
organizational  analysis  fail  in  achieving  high  performance  validity.  Turn 
to  Page  40. 

(2)  Since  performance  validity  is  concerned  with  the  transfer  of  learning 
to  another  environment,  you  must  know  something  about  that  environment. 
Turn  to  Page  63. 

(3)  Because  there  are  system-wide  components  in  an  organization  which 
affect  performance  and  are  not  Included  in  task  or  trainee  characteristic 
assessment.  Turn  to  Page  89. 

(4)  All  of  the  answers  listed  here.  Turn  to  Page  98. 
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(3)  Right  on.  Each  hazard  discovered  should  first  be  grouped  according  to 
its  severity  ranking. 


The  first  thing  you  would  do  in  a  hazard  analysis  is  to  fill  out  a 
data  form  such  as  Figure  27.3  for  each  hazard  identified.  Next,  you  would 
rank  each  hazard  or  group  the  hazards  according  to  their  severity.  Now  you 
can  continue  the  evaluation  by  analyzing  similarly  ranked  hazards.  If,  for 
example,  there  were  no  observed  hazards,  and  hazards  A,  B,  and  C  had  all 
received  a  rating  of  catastrophic,  what  would  your  next  step  consist  of? 

(1)  Ranking  these  hazards  (A,  B,  and  C)  according  to  their  probability  of 
occurrence.  Turn  to  Page  93. 

(2)  Determining  the  cost  of  corrective  action  for  each  hazard  (A,  B,  and 
C).  Turn  to  Page  40. 

(3)  Reevaluating  these  hazards  (A,  B,  and  C)  until  there  were  no  tie 
ratings.  Turn  to  Page  59. 


From  Page  17 


(2)  This  is  not  correct.  We  suggest  that  you  review  the  material  in  Lesson 
3.  Return  to  Page  17. 


From  Page  30 


(1)  A  difference  threshold  is  one  of  the  types  of  values  that  we  will  be 
discussing,  but  the  present  example  doesn't  depict  this  type.  Return  to 
Page  30. 
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(1)  Very  good,  you  are  exactly  correct.  Keep  up  the  good  work. 

After  you  have  grouped  the  hazards  according  to  severity  and 
probability  of  occurrence,  the  next  step  Is  to  evaluate  according  to  costs. 
To  keep  to  our  hazard  exaaple,  suppose  after  you  ranked  hazards  A,  B,  and  C 
for  their  probability  of  occurrence,  you  found  that  hazards  A  and  B  were 
Imminently  probable.  Then  you  were  Inforaed  that  the  costs  required  to  fix 
hazard  A  were  far  wore  than  the  budget  allowed.  You  would  rate  hazard  A ' s 
costs  as  prohibitive.  Hazard  B,  however,  would  only  cost  a  minimal  amount. 
What  action  would  you  recommend  for  hazard  A? 

(1)  Take  Immediate  action.  Turn  to  Page  57. 

(2)  Defer  action.  Turn  to  Page  45. 

(3)  Analyze  further.  Turn  to  Page  73. 


Prom  Page  11 


(1)  This  lsn*t  the  first  grouping.  There  Is  a  more  Important  category  than 
cost.  Return  to  Page  11. 


Prom  Page  38 


(1)  It  would  be  unlikely  that  the  number  of  misses  would  Increase  even 
through  the  false  alarms  probably  would.  Return  to  Page  38. 


From  Page  72 


I _ I 

(1)  Exactly  right.  Describing  those  aspects  of  a  job  which  are  critical 
for  job  success  is  the  main  purpose  of  a  job  analysis. 

A  job  analysis  produces  a  job  description.  For  example,  for  the 
operator  of  our  coastal  defense  system,  the  job  description  might  include 
the  following:  operates  air  search  radar  for  detection  of  enemy  aircraft 
and  missiles,  directs  the  use  of  antiair  weapons,  tracks  air  targets, 
identifies  targets  as  hostile  or  friendly,  etc. 

The  third  analysis  of  this  series  has  been  called  person  analysis  by 
some  researchers.  However,  we  think  you  will  recognize  this  third  training 
analysis  as  task  analysis,  which  was  discussed  in  Lessons  23  and  24.  In 
doing  this  analysis  the  job  requirements  must  be  translated  into  human 
attributes  required  to  perform  given  tasks.  One  facet  of  this  analysis  is 
an  examination  of  the  performance  capabilities  of  your  available 
population.  You  must  know  whether  or  not  the  required  behavioral 
characteristics  are  already  within  the  individual's  capabilities. 

Let's  talk  about  actual  procedures  that  you  as  a  human  factors 
specialist  would  follow.  Remember,  you're  trying  to  identify  the  people 
who  will  operate  this  system  and  determine  how  best  to  select  and  train 
them. 


As  a  human  factors  specialist  you  will  draw  information  from  many 
sources — from  the  systems  design  team,  from  mock-ups,  from  prototype  tests, 
and  from  field  experience.  In  addition,  there  is  a  series  of  documents  you 
will  need  to  examine  to  initiate  your  staffing  study.  Figure  28.1 
summarizes  the  following  discussion. 

There  are  four  basic  documents  with  which  you  will  be  concerned: 
engineering  drawings,  operational  and  maintenance  scenarios,  reliability 
analyses,  and  operation  manuals.  These  documents  are  referred  to  as  basic 
source  documents. 

Engineering  drawings  provide  the  human  factors  specialist  with 
information  about  location  and  dimensions  of  work  stations,  the  nature  of 
the  work  environment,  and  other  valuable  information  which  allows  him  to 
make  Inferences  about  the  duties  and  sequence  of  tasks  an  operator 
malntalner  will  follow  when  Involved  with  a  system.  If  these  drawings 
indicate  probable  trouble  spots,  the  human  factors  specialist  must  propose 
solutions  (using  the  same  alternatives  as  shown  in  DI-H-7058)  at  this  stage 
to  prevent  later  problems. 


(GO  ON  TO  THE  NEXT  PAGE) 
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From  Page  94 


The  second  source  docuaent  a  human  factors  specialist  needs  to  examine 
Is  the  operational  and  aaintenance  scenario.  This  document  will  detail  how 
the  systea  will  be  used  and  aaintained  when  it  is  put  into  operation.  The 
variations  in  systea  load  will  be  addressed.  What  will  the  impact  of  war 
be  on  the  systea  and  its  operators  and  aalntalnera?  Will  there  be  certain 
peak  periods?  If  so,  how  can  we  provide  for  this? 

If  you  were  a  military  manpower  analyst,  how  would  you  provide  for 
peak  periods? 

(1)  By  aannlng  systems  with  people  capable  of  using  them  in  a  wartime 
scenario.  Turn  to  Page  9. 

(2)  By  providing  the  alniaua  number  of  personnel  required  for  peacetime 
operations  and  increasing  the  training  of  these  people.  Turn  to  Page  59. 

(3)  Since  peak  periods  occur  so  infrequently,  it  is  not  cost  effective  to 
aan  a  systea  this  way.  Turn  to  Page  38. 


(3)  A  difference  level  is  established  by  using  the  classic  psychophysical 
methods,  not  the  direct  scaling  aethods.  Return  to  Page  41. 


Froa  Page  74 


(2)  Right.  By  asking  the  old  stand-by  questions  'What  are  the  limitations 
and  capabilities  of  the  huaan?'  and  'What  are  the  huaan  perforaance 
requlreaents? '  Huaan  factors  specialists  discovered  that  the  operations  of 
this  systea  might  be  required  to  escape.  However,  the  huaans  inside  the 
structure  didn't  have  the  capabilities  to  do  so.  The  situation  was  soon 
reaedled. 

Well,  we  once  again  cone  to  the  end  of  a  lesson.  You've  learned  about 
hazard  analysis  and,  probably,  have  acquired  a  'hazard  awareness'  in  the 
process.  We  hope  so,  anyway.  At  least  I.  H.  Eager  did.  After  taking  this 
lesson,  he  Immediately  went  over  his  design  for  the  super  helo  and 
reanalyzed  It  for  safety.  He  discovered  that  while  his  helo  had  land,  air, 
and  sea  capabilities.  It  didn't  have  an  air-lock.  Therefore,  his  sea 
capabilities  would  have  required  fish  for  crew  members,  because  humans 
would  have  been  up  to  their  neck,  so  to  speak.  In  water. 

See  you  next  lesson  where  you  will  learn  everything  you  ever  wanted  to 
know  about  training  but  were  afraid  to  ask. 


Froa  Page  58 
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(2)  The  workers  aost  reaoved  from  day-to-day  maintenance  operations  possess 
a  considerable  amount  of  expertise  because  they  handle  the  aost  technically 
sophisticated  aalntenance.  Return  to  Page  58. 


Froa  Page  63 


(2)  You're  way  off.  Physical  fidelity  is  quite  often  important  whether  you 
are  dealing  with  part  or  whole  simulation.  Return  to  Page  63. 


Froa  Page  66 


(2)  The  workers  aost  reaoved  froa  day-to-day  maintenance  operations  possess 
a  considerable  amount  of  expertise  because  they  handle  the  most  technically 
sophisticated  maintenance.  Return  to  Page  66. 


(4)  Very  good.  The  main  point  here  Is  that  you  oust  examine  all  variables 
If  you  desire  success. 

Assisting  we  account  for  the  organizational  variables ,  our  next  concern 
is  lntra-organlzatlonal  validity.  This  type  of  validity  assumes  that  we 
have  established  training  and  performance  validltiy,  and  are  now  concerned 
with  predicting  the  perforaance  of  a  new  group  of  trainees.  This  is  a 
primary  goal  of  aany  corporate  training  programs.  This  is  ordinarily  the 
level  of  generalization  desired.  There  are  several  factors  which  must  be 
considered.  First,  you  aust  be  certain  that  the  various  components  of  your 
needs  assessment  have  not  changed.  You  must  also  be  certain  that  your 
training  program  has  not  changed.  Finally,  the  value  of  your 
generalization  from  perforaance  validity  to  lntra-organlzatlonal  validity 
rests  on  how  well  your  original  evaluation  was  conducted.  It  should  be 
clear  that  the  higher  the  quality  of  the  training  and  performance  validity, 
the  easier  it  Is  to  generalize. 

Imagine  the  following  situation.  A  corporation  has  developed  a 
training  program  for  its  accountants.  Training,  performance,  and 
lntra-organlzatlonal  validity  have  been  established.  You  are  In  business 
for  yourself  (not  the  same  business  as  the  corporation)  and  you  feel  you 
need  a  training  program  for  your  accountants.  Do  you  think  it  wise  to 
generalize  the  validity  of  the  corporation's  training  program? 

(1)  No,  since  the  needs  assessment  and  evaluation  have  not  been  performed 
for  your  organization.  Turn  to  Page  86. 

(2)  No,  since  the  tasks  iriilch  need  to  be  taught  are  different.  Turn  to 
Page  48. 

(3)  Yes,  since  the  tasks  requirements  are  essentially  the  same.  Turn  to 
Page  59. 

(4)  Yes,  although  you  must  take  Into  account  any  difference  between  the  two 
organizations.  Turn  to  Page  7. 


(3)  While  you  are  partially  correct  In  assuming  that  this  answer  Is 
Important  in  designing  for  maintainability,  it  is  not  the  best  answer 
provided  here.  Go  back  to  Page  88. 


From  Page  11 


(2)  This  isn't  the  first  grouping  which  needs  to  be  made.  There  is  another 
category  which  Is  of  higher  priority  than  the  probability  one.  Return  to 
Page  11. 


From  Page  58 


(2)  This  is  not  the  right  answer.  If  you  are  guessing.  It  might  be  a  good 
Idea  to  go  back  to  Lesson  16.  Return  to  Page  58. 


Fro*  Page  74 


(4)  Nothing  in  this  situation  indicated  that  training  or  expertise  was 
deficient.  There  is  a  structural  problen  here,  not  a  personnel  one. 
Return  to  Page  74. 


Fro*  Page  41 


(2)  This  is  the  rationale  behind  the  estimation  methods.  The  production 
methods  require  more  active  participation  on  the  observer's  part.  Return 
to  Page  41. 


•  U.8.  QOVENMEKT  PRIMTIM  OfFICE  i  1*«2  0  -  m-t«7  (Vol.t) 


